Substack shipped an AI detector this week. Every post, note, and comment over 100 words can now be scanned through Pangram to see how much of it reads as human or AI. Chris Best's launch post frames it as giving readers a choice: the platform isn't banning AI use, just surfacing it. Worth knowing going in: Pangram's own data already ranks Substack as the cleanest of the platforms it scans, a fraction of LinkedIn's AI-content rate. They're launching transparency tooling from the platform with the least to hide.
I read that and thought: I already know exactly how this goes. I lived it on a smaller scale, with a person instead of a platform integration.
A few months ago I got flagged twice in one day by "Sloan," DEV.to's moderation-warning system. Not a bot quietly scoring posts in the background. A specific community member, reading articles and running them through GPTZero, then sending the same message a blunt classifier would have sent.
The two pieces that got flagged were the ones that generated the most technical discussion I'd published all year. Short paragraphs. Named data points. Rhetorical questions doing real work. The features that make an argument land are the same features that read as "AI-shaped" to anyone calibrated to notice them, human or model.
Write worse, look more human. Write well, get flagged.
That thread also surfaced the part nobody had a clean answer for: the policy creates a dishonesty incentive. Two equally AI-assisted pieces, equally good — the one with a disclosure gets flagged, because now there's something to catch. The one without doesn't. The system was catching transparency, not AI use.
And then Marco showed up in the comments. Forty years in tech, writing in his second language, using AI to make sure his Italian didn't flatten into something stiffer than he meant. Same Sloan message. Same classifier verdict. Nothing to do with what the policy was built for.
Pangram is a real classifier with real engineering behind it: hard negative mining against its own false positives, training data deliberately mirrored so it can't just learn "formal writing = AI." That's more rigor than one guy running GPTZero between article reads. I'll give it that.
But it inherits the same structural problem Sloan had, because it's answering the same narrow question: does this text look AI-shaped. Not: did a human do the thinking. Chris Best's own post admits as much. Pangram can't tell you whether care went into something, only whether the sentences pattern-match to a machine's output.
That gap is where Marco lives. Detectors trained without deliberately mirrored data have a documented habit of flagging non-native English writing, since careful, formal phrasing correlates with both AI output and someone translating in their head before they type — enough of a problem that several major universities have stopped letting instructors use AI detectors at all. Pangram claims their mirror-prompt method fixes it. Maybe. Most of the numbers backing that claim trace back to Pangram or a study Pangram commissioned.
Someone with no stake in the answer already looked. The Atlantic's Matteo Wong traced a recent wave of AI-writing accusations back to Pangram itself, including a horror novel pulled from a major publisher days before its release. His argument wasn't that the tool is broken. It's that a detector that's mostly reliable can be more dangerous than one that's obviously unreliable, because people stop checking. A 99.98% accuracy rate sounds like certainty. Applied across millions of posts, the failures are still real people, still real reputations, just quieter about it.
That's Marco's risk, and mine, in one sentence: the false positive doesn't feel like a statistic when it's your byline.
I write from Port Harcourt, in English, the language I was taught in and think in, using AI as part of an actual workflow: not to generate opinions I don't have, but to get from a rough draft to a clean one without losing the argument along the way. Sloan already showed me what a false positive costs, close enough that I don't need to imagine it. Marco is the version of that risk I can't unsee.
It's also the whole reason I stopped using a generic humanizer and built one calibrated to my own published corpus instead. A tool trained to strip "AI-shaped" patterns from anyone's writing will also strip the parts of your writing that are just yours, an em dash you use structurally, a habit of compressing three examples into two. Voice-humanizer checks against what I actually sound like, not against a mirrored dataset of nobody in particular.
Sloan and Pangram are both answering "does this look like AI." I don't think that's the question that matters. The question is whether someone can be asked "did you know what you were writing about, and do you stand behind it," and answer yes.
I do.
