Run older, canonical human-written texts through AI detectors and they can still come back as AI-generated. That tells you something the vendors' marketing pages do not: the tool is not measuring what its name claims to measure.
That gap is the whole subject. An AI text detector does not detect AI authorship. It detects a type of writing, text whose statistical fingerprint resembles the AI output the detector was trained on. When human writing happens to share that fingerprint, the detector flags it, and it cannot tell the difference. The people who lose scholarships, contracts, and academic standing over a percentage score are on the wrong side of that confusion.
This is what the score represents, why the errors are structural rather than a bug the next version will fix, who they fall on, and what a serious institution can use instead. That is why AI detectors are wrong in the only setting where the question really matters: high-stakes decisions about real people.
The Short Version
- AI detectors measure statistical resemblance, not authorship. A high "AI" score means the text looks like the writing the detector was trained to associate with AI. It does not establish that a machine produced it, and it cannot.
- The false-positive rate is a mathematical floor, not an engineering bug. A 2026 mathematical framing argues that any useful text-only, one-shot detector faces false accusations at a rate set by how much human writing and AI writing overlap. Better models do not remove it.
- The errors fall hardest on disciplined writers. Non-native English speakers and writers in disciplined, constrained styles, including scientific, legal, and technical writers, get flagged more often, because clean, predictable prose shares the statistical profile detectors read as "AI."
- Provenance is the approach replacing statistical detection. Watermarking (SynthID) and signed content credentials (C2PA) record origin at generation time instead of guessing at it afterward. That is verifiable, but only for content the compliant tools touched.
What This Article Doesn't Cover
- It is not a ranking of which detector is "best." The argument here is that the ranking question is the wrong one.
- It is not a guide to evading detection. There are plenty of those elsewhere; this is about what the measurement means.
- It is not legal advice for a specific accusation. If you are contesting one, talk to someone who handles that.
- It is not a tutorial. There is no tool to install and no configuration to copy.
What Are AI Detectors Actually Measuring?
An AI text detector measures how closely a piece of writing resembles AI-generated text, using three signals: perplexity, burstiness, and trained classifiers. It reports a probability that the text is machine-written. What it does not do, and structurally cannot do, is verify who or what produced the text. It inspects the words and infers, never the process that created them.
Perplexity is how "surprised" a language model is by the next word in a sequence. AI text tends to pick the statistically most probable next token at each step, which reads as low perplexity. Human writing takes stranger turns and scores higher. Burstiness measures variation in sentence length and structure. Humans mix short and long; AI tends toward uniform. A detector combines the two into a composite score. The trained-classifier approach skips the hand-picked signals and instead learns latent patterns from large datasets of labeled human and AI text.
Each signal has the same flaw, wearing different clothes. Burstiness cannot separate "disciplined human writer" from "AI"; a legal brief and a chatbot both produce low variance. Classifiers trained on one model's output do not carry over to the next. The RAID benchmark, one of the largest published evaluations of AI-text detectors, tested more than 6 million generations across 11 models, 8 domains, 11 adversarial attacks, and 4 decoding strategies. Its core finding is that detectors are easily weakened by adversarial attacks, sampling changes, repetition penalties, and unseen generators. And perplexity has a precision problem that is worth stating plainly.
The proxy-model problem. To measure a text's perplexity accurately, you need the full probability distribution (the logits) from the exact model that wrote it. Detectors almost never have that; they estimate perplexity with a proxy model instead. When the writing model and the measuring model differ, which is the normal case, the perplexity number carries systematic error baked in from the start. The most sophisticated statistical method to date, Binoculars, reduces that noise by comparing two related models' perplexity, and it is still measuring the statistics of the text, not the origin of it.
That last clause is the point of the whole section. Every method here, from a crude perplexity threshold to Binoculars, is reading the properties of the words. None of them observes the act of writing. They measure resemblance to a training distribution.
Resemblance is not authorship, that is the entire problem, in five words.
Why Do AI Detectors Produce So Many False Positives?
Detectors flag human writing as AI because they flag any writing whose statistical properties resemble AI output. A 2026 mathematical framing treats this as more than a tuning error: when the assessor does not know each person's individual writing distribution, false accusations are unavoidable, at a rate set by how much human and AI writing overlap. The floor is real, and it does not move.
The paper is Garland's 2026 "AI Detectors Fail Diverse Student Populations: A Mathematical Framing of Structural Detection Limits." Conventional detection theory treats the task as a test between two known distributions: this is what human writing looks like, this is what AI writing looks like, decide which one produced the text. Garland's argument is that the human side is not one distribution. Every person's natural style is its own distribution, and some people's styles overlap heavily with AI output. In statistical terms the null hypothesis is composite (a bundle of many distributions rather than a single one), and a text-only, one-shot detector working against a composite null has no way to avoid false accusations.
"Any text-only, one-shot detector with useful power must produce false accusations at a rate governed by the distributional overlap between student writing and AI output." Garland, 2026 (arXiv:2603.20254)
The consequence is worth being precise about, because it is what separates this from the usual "detectors aren't perfect yet" framing. The bound comes from population diversity, not from model quality. A better detector, a bigger training set, a smarter classifier: none of them touch it, because the overlap it depends on is a property of how people write, not of how well the tool is engineered. Garland's own policy line follows directly: "detection scores should not serve as sole evidence in misconduct proceedings."
The empirical record lines up with the math. OpenAI built a classifier for its own models' output, watched it identify AI text only 26% of the time while falsely flagging humans 9% of the time, and shut it down in July 2023, citing its low reliability "given that educators could be making judgments about students with potentially lasting consequences." A 2026 peer-reviewed study in the International Journal of Educational Integrity put Turnitin at 61% and Originality.ai at 69% real-world accuracy on a mixed dataset, a long way from the 99% on the marketing pages. An hCaptcha benchmark concluded that no public detector it tested beat random chance. Curtin University later disabled Turnitin's AI writing detection feature from January 1, 2026, citing the need for trust, clarity, fairness, and future-ready assessment.
Then multiply by scale. Vanderbilt disabled Turnitin's detector after doing the arithmetic on its own volume: at a claimed 1% false-positive rate across 75,000 annual submissions, roughly 750 students a year would be wrongly flagged. That is the low estimate, taken from the vendor's own optimistic number.
The false-positive rate is a floor set by how differently people write, not a ceiling the next release lowers.
Who Gets Falsely Flagged the Most?
Yes, detectors are biased, and systematically. Non-native English writers and writers in disciplined, constrained styles (legal, scientific, technical) are flagged more often because their writing can carry the low-perplexity, low-burstiness profile detectors read as "AI." The bias is not about who they are; it is that careful, plain, predictable prose can look statistically like machine output.
The foundational evidence is Liang et al.'s 2023 study in Patterns. Seven widely used detectors were run against 91 TOEFL essays from non-native English speakers and 88 US eighth-grade essays from native speakers. The detectors misclassified more than half the non-native essays (an average false-positive rate of 61.3%) while scoring near-perfect on the native-speaker set. All seven detectors unanimously flagged 19.8% of the human-written TOEFL essays as AI-authored.
The experiment that closes the case is the intervention. When the researchers used ChatGPT to enrich the vocabulary of the same non-native essays to sound more native-like, the false-positive rate fell from 61.3% to 11.6%. Making the text more AI-touched made the detectors flag it less, because what they were reacting to all along was vocabulary predictability, not authorship. The signal driving the accusation was perplexity, and perplexity was never a measure of who wrote the words.
The pattern does not stop at language proficiency. BAID, the first systematic bias benchmark, evaluated detectors across seven sociolinguistic axes (demographics, age, grade level, dialect, formality, political leaning, and topic) over 200,000+ samples, and found consistent disparities on all seven. Rashidi et al. found that an AI text detector misidentified up to 8% of known human-written scientific abstracts as AI-generated, using abstracts published between 1980 and 2023, because medical and scientific writing runs on constrained vocabulary, hedged phrasing, and standardized structure. Legal writing is formulaic by design. The Authors Guild put the professional-writer version of this plainly: the more refined and controlled a writer's style, the more it resembles the output these tools were built to flag.
The writers most likely to be falsely accused are the ones who write in the most disciplined, constrained ways, the exact opposite of what "cheating" would predict.
If Detectors Work, Why Can Anyone Bypass Them?
Bypassing a detector is routine, not clever. Detectors already run low, and adversarial manipulation drives them lower still; adversarial paraphrasing cuts detector true-positive rates by an average of 88%. The arms race is asymmetric by construction: a detector must defend every evasion path at once, while a bypass tool only has to beat the single pattern the detector currently measures.
The numbers come from the research directly. Perkins et al. (2024) measured detector accuracy at 39.5% on machine-generated text, falling to 17.4% once evasion techniques were applied. Cheng et al. (2025) found adversarial paraphrasing reduced average true-positive rates by 87.88% across detector types, and cut Fast-DetectGPT by 98.96%. Sadasivan et al. (2023) showed that recursive paraphrasing can sharply reduce detector performance, including for watermarking-based detectors, while keeping the text readable. Around these findings sits an entire counter-industry of "humanizer" tools whose job is to rewrite AI text until it scores as human, and the existence of that industry is itself evidence about what detectors measure. You cannot build a reliable tool to defeat a measure of authorship. You can build one to defeat a measure of text statistics, and people have.
The asymmetry is structural, and it shows in the release cadence. When Turnitin shipped an AI-bypasser detection feature in August 2025, an attempt to catch text that had been run through humanizers, humanizer vendors quickly began advertising bypass claims of their own. Every detector update defines a new target; every target gets hit.
There is an inference the reader can draw from all of this, and it is worth marking as an inference rather than a fact. Read this way, the detectors are mostly catching people who submit raw, unedited AI output: the least motivated and least careful users. The ones a policy most wants to catch are the ones most easily missed.
The arms race is not a temporary gap the vendors will close. It is asymmetric by design.
What Are Institutions Doing Now?
A growing list of universities (Vanderbilt, Yale, Curtin, the University of Waterloo, and more) have disabled or restricted Turnitin's AI detector, citing false-positive volume, bias against non-native speakers, unstable scores, and a lack of transparency. Others kept it only as an advisory signal, never the sole basis for an accusation. The institutional verdict is arriving independently of the academic papers, and it agrees with them.
The reasoning is documented and specific. Vanderbilt named four grounds when it disabled the feature in August 2023: the 750-false-accusations-a-year arithmetic, the bias against non-native speakers, the absence of any explanation of how Turnitin reaches its verdict, and privacy concerns about third-party data submission. Curtin University announced that, from January 1, 2026, Turnitin's AI writing detection feature would be disabled across all campuses and study periods, while regular text-matching checks would remain active. The University of Waterloo discontinued Turnitin's AI detection functionality from September 2025 after internal academic consultation. The University of Texas at Austin does not endorse AI detection software, has no central contracts or purchase orders with active AI-detection features, and classifies this software as high risk for procurement. Faculty guidance from institutions including MIT and Stanford lands on the same practical lesson: AI detectors have high error rates, false positives, and bias risks, so they should not be treated as decisive evidence.
Underneath the policy language are people. Marley Stevens, a student at the University of North Georgia, was flagged by Turnitin on work she wrote herself, placed on academic probation, and lost her HOPE Scholarship; she says she had only run the text through Grammarly. At UC Davis, one student accused of using AI was later cleared after showing Google Docs edit history, and a separate student-run test reported that GPTZero falsely flagged 40% of 247 non-AI documents. These are not the edge cases the error rate rounds off. At the volumes these tools run at, they are the error rate made visible.
What Replaces Statistical Detection?
The emerging answer is provenance: instead of inspecting finished text and guessing at its origin, record a verifiable signal of origin at the moment of generation. Two approaches are converging, Google DeepMind's SynthID watermarking and the C2PA Content Credentials standard, paired with older evidence like draft history and in-class work. Provenance does not guess better. It changes the question to one that can be answered.
SynthID works by nudging the token probabilities as a model generates text, leaving a statistical pattern that a verifier can later check for. Google has deployed SynthID across generated images, text, audio, and video; its image and video-frame implementation has been used to watermark over 10 billion images and video frames, and Google now provides a SynthID Detector portal for supported media. Its limits are documented: it works best on longer, varied outputs, performs poorly on short or purely factual answers (there is only one right way to write France's capital, so there is nothing to modulate), and its confidence degrades under heavy rewriting or translation. It also cannot see text from any model that does not implement it.
C2PA Content Credentials take the complementary approach: cryptographically signed metadata attached at creation time, recording what tool made the content and when. OpenAI joined the C2PA steering committee in May 2024. In May 2026, it expanded its provenance stack for supported image outputs by pairing C2PA Content Credentials with Google DeepMind's SynthID watermarking and previewing verification tooling. The two layers back each other up. Signed metadata is rich but can be stripped on re-upload, while a SynthID watermark survives screenshots and format changes but carries less information. The catch is the same one that limits every provenance scheme: it verifies content from tools that participate, and says nothing about content from tools that do not. Coverage is voluntary, and it grows only as adoption grows.
Which is why the field does not stop at watermarks. The alternatives that university guidance and community consensus keep landing on are procedural: require draft history and version commits, build in short in-class or oral components, and design assessments that are hard to fake without genuine engagement. And when a signal does turn up, treat it as the opening of a conversation, not the close of a case.
That is the concrete takeaway an evaluator can carry to stakeholders. Statistical detection asks "does this text look like AI?", a question that, per Garland, has no reliable answer. Provenance asks "did a compliant tool sign this?", a question that has a verifiable one, for the subset of content those tools touched. The trade is narrower coverage for a claim you can actually stand behind, which is the better position to be in when a person's standing is on the line.
Marketed Accuracy vs. Independent Findings
Vendor accuracy claims and independent measurements are not close. The table below sets each tool's marketed number against what independent testing found. It is not a buying guide; there is no "recommended" column, because the argument of this article is that the framing behind such a column is broken. It is a record of the gap.
| Tool | Vendor-marketed accuracy / FPR | Independent finding |
|---|---|---|
| GPTZero | 99% accuracy, 1% false-positive rate | 16% false-positive rate on human-written essays in a 78-essay study |
| Turnitin | <1% false-positive rate | 61% overall accuracy in a 2026 International Journal for Educational Integrity study |
| ZeroGPT | 98.5% detection accuracy | 83% false positives on human-written medical abstracts in a foot-and-ankle-surgery study |
| Originality.ai | 99%+ accuracy / low false-positive claims, depending on model | 76% overall accuracy in Scribbr's 2024 review; 69% overall accuracy in a 2026 academic-context study |
| Copyleaks | Over 99% accuracy | Accuracy fell to 71% on humanized DeepSeek-generated text in a 2025 detector study |
| OpenAI classifier | N/A | 26% true-positive rate, 9% false-positive rate; discontinued July 20, 2023 because of low accuracy |
These figures are not directly comparable as benchmark scores because each test used different datasets, thresholds, and writing conditions. The point is the recurring gap between controlled vendor claims and messier real-world or independent evaluations.
Frequently Asked Questions
What Does an AI Text Detector Actually Detect: AI, or a Type of Writing?
It detects a type of writing. A detector measures whether text statistically resembles AI output: low perplexity and low burstiness, or a match to a trained classifier's learned patterns. It cannot verify authorship. A high score means the writing looks like the AI text the tool was trained on, not that a machine produced it.
Why Did My Human-Written Essay Get Flagged as AI-Generated?
Because your writing shares the low-perplexity statistical profile detectors read as AI, a profile common in polished, technical, or non-native-English writing. The detector reacts to predictable vocabulary and uniform sentence structure, not to authorship. A flag is a statement about your text's statistics, not evidence that you used AI.
Are AI Detectors Biased Against Non-Native English Speakers?
Yes, measurably. Liang et al. (2023) found an average false-positive rate of 61.3% on TOEFL essays from non-native writers, versus near-zero on native-speaker essays. The BAID benchmark later found similar disparities across seven axes including dialect, formality, and topic. The cause is statistical: constrained vocabulary reads as low perplexity, which detectors misread as AI.
Why Does the Same Text Get Different AI-Detection Scores on Repeated Scans?
Because detector scores are model-based estimates, not direct observations of authorship. Thresholds, classifier behavior, preprocessing, and tool updates can all affect the final percentage, so a score should be treated as a weak signal rather than a stable measurement.
What Should Organizations Use Instead of AI Text Detectors?
Provenance tools (SynthID watermarking and C2PA Content Credentials) for content from compliant generators, paired with process evidence like draft history, version commits, and in-class work, plus assessments redesigned to require genuine engagement. Any detector output should start a conversation, never serve as sole evidence in a decision that affects someone's standing.
