AI Detection in 2026: How It Actually Works, When It Fails, and What the Research Says

In August 2023, Vanderbilt University turned off Turnitin's AI detector. Not because it didn't work — because of what happens when it half-works at scale. Vanderbilt ran the arithmetic in public: Turnitin claimed a 1% false positive rate, and Vanderbilt had submitted roughly 75,000 papers the previous year. One percent of 75,000 is about 750 students who could have been wrongly told a machine wrote their essay.

That is the whole problem with AI detection in one calculation. The technology is real, the signals it measures are real, and at institutional scale a "small" error rate becomes hundreds of accused people.

AI detection is statistical inference, not proof. A detector measures whether text has the statistical fingerprints typical of language-model output — mainly how predictable the word choices are and how much sentence length varies — and returns a probability. It cannot see who typed the document, and it produces no evidence in the forensic sense. Every responsible use of a detector starts from that sentence.

This guide covers what the signals actually are, what the peer-reviewed research says about accuracy, where detection breaks down, and how to read a result without doing harm.

What is an AI detector, exactly?

An AI detector is a classifier. You give it text; it estimates the probability that a large language model produced it.

It is emphatically not a plagiarism checker. Plagiarism detection is a search problem — take this document, look for matching passages in a corpus of existing documents, report overlaps. That's a lookup, and its output is verifiable: here is the source, here is the match.

AI detection has no corpus to check against. There is no database of every sentence ChatGPT has ever written. So detectors instead ask a statistical question: does this text behave the way model-generated text behaves?

That difference matters enormously for how much weight a result deserves. A plagiarism hit points at a specific source you can go read. An AI detection score points at nothing. It is a probability estimate about the process that produced the words.

The four signals detectors actually measure

Perplexity: how predictable are the word choices?

Perplexity, in this context, measures how surprised a language model is by each next word in a text. Low perplexity means the words are highly predictable — each one is roughly what a model would have guessed.

This is the single most-used detection signal, and the reason is structural. Language models generate text by repeatedly picking high-probability next tokens. That is literally the mechanism. So model output tends to sit in the statistically "safe" middle of the distribution.

Humans wander. We pick the odd word, the slightly wrong-but-better word, the word that only makes sense because of something we were thinking about three sentences ago.

The catch — and this is the crux of the entire fairness problem — is that writing simply also produces low perplexity. More on that below, because it's where detection does its real damage.

Burstiness: does sentence length vary?

Burstiness measures variation in sentence length and structure across a passage.

Human writing is bursty. We write a long, clause-heavy sentence that develops an idea across several turns and then qualifies it. Then a short one. Then a fragment.

Model output tends toward uniformity — sentences of similar length, similar construction, similar rhythm. Not identical, but noticeably more regular than most human prose.

Vocabulary diversity

How much does the vocabulary range vary across the passage? Models often draw from a somewhat narrower effective vocabulary than a human writing on the same subject, and — more tellingly — they distribute that vocabulary more evenly.

Model-specific phrasing patterns

Different model families have recognizable habits. Claude qualifies claims in a particular way. Gemini has a characteristic approach to structuring definitions. GPT-4o and GPT-5 have their own recurring constructions.

A detector that maintains per-model fingerprints can be calibrated separately for each, which matters because — as anyone who has tested this will tell you — the same detector scores Claude output substantially lower than ChatGPT output on identical prompts. A single universal threshold under-detects some models and over-flags others.

If you want the model-by-model detail, we've written it up separately for ChatGPT, Claude, and Gemini.

What does the research actually say about accuracy?

Here is where most articles on this topic quietly stop citing anything. So let's be specific about what has actually been measured.

The Stanford study is the one that matters

In 2023, Weixin Liang, Mert Yuksekgonul and colleagues published GPT detectors are biased against non-native English writers in Patterns (Cell Press, Article 100779). They ran seven commercial GPT detectors against two sets of human-written text: essays by US eighth-graders, and TOEFL essays written by non-native English speakers under supervised exam conditions.

The detectors identified the US student essays with near-perfect accuracy.

They classified more than 61% of the TOEFL essays as AI-generated.

Every one of those essays was written by a human, in an exam room, under supervision. The detectors were not detecting AI. They were detecting non-native English.

The researchers then did something clever that confirmed the mechanism: they used a language model to improve the word choice in the TOEFL essays, and misclassification dropped. Then they simplified the US eighth-grade essays, and misclassification rose. The detectors were tracking linguistic sophistication, and calling anything below a threshold "machine."

That is not a bug in one product. It is a consequence of using perplexity as your primary signal, and it affects every detector that does — including, to be honest about it, ours.

Has anyone solved it?

Partially, and it's worth acknowledging. Pangram Labs publishes ESL false-positive testing across four datasets totalling 25,021 samples (ELLIPSE, ICNALE, PELIC, and the Liang TOEFL set), reporting an aggregate false positive rate of 0.012% — and 0% on the Liang TOEFL essays specifically. They report Turnitin at 1.4% on ESL text over 300 words against their own 0.02%, and GPTZero at 1.1% on the Liang set against their 0%.

Those are vendor-published figures on their own product, so weigh them accordingly. But the datasets are public and named, which is more methodological transparency than most of this industry offers.

The honest summary: the ESL bias problem is well-documented, it is not universal across all tools, and it has not gone away.

Where detection reliably fails

Short text. Below roughly 50 words there simply isn't enough signal. Perplexity and burstiness are distributional measures; they need distribution. Any tool returning a confident verdict on two sentences is performing confidence, not measuring it.

Edited text. This is the big one, and it's underappreciated. AI output that a human has revised — reordering clauses, swapping words, cutting sentences — loses its statistical signature quickly. Not because the human hid it deliberately, but because human editing reintroduces human irregularity. A lot of real-world writing is exactly this hybrid, and detectors handle it poorly.

Paraphrased and humanized text. Tools built specifically to defeat detectors work by increasing perplexity and burstiness artificially. Detection confidence drops. Sometimes it drops to useless. We built a humanizer bypass detector that screens for the residual patterns these tools leave, and it helps — but anyone claiming reliable detection of well-humanized text is overselling.

Non-native English writing. Covered above. This is the failure mode with actual human consequences, and we've written about it in detail.

Brand-new models. When a new frontier model ships, its statistical fingerprint is unknown to every detector on the market. There's a lag. During that lag, detection on its output is measurably worse.

Highly formulaic human writing. Legal boilerplate, technical documentation, lab reports, structured business writing. These are low-perplexity by genre convention. A detector doesn't know the difference between "predictable because a machine wrote it" and "predictable because that's how a contract is written."

How to read a detector result responsibly

A percentage is not a verdict. Here's what to do with one.

Treat sentence-level output as the actual result. A document-level score of 68% tells you almost nothing actionable. Which sentences drove it? If flagged passages cluster in one section, that's a different situation from flags scattered evenly — the former often means a specific chunk was pasted in, the latter often means the whole document is being penalized for style.

Check the length. Under 50 words, disregard the number entirely.

Ask who wrote it. If the author is a non-native English speaker, the base rate of false positives goes up substantially — 61% in the Liang study. This should change how much weight you give the score, not just how you phrase your follow-up.

Consider the genre. Formulaic writing scores high for reasons unrelated to AI.

Never use a score alone to make a decision about a person. This is the one that matters. A detector result is a reason to have a conversation — ask about the drafting process, look at revision history, discuss the sources. It is not evidence of misconduct, and Vanderbilt's decision to disable their detector was fundamentally a recognition that treating it as evidence causes more harm than the cheating it catches.

Quick reference

Question	Answer
What does an AI detector measure?	Statistical properties of the text — mainly word predictability (perplexity) and sentence-length variation (burstiness) — not the identity of the author.
Is AI detection the same as plagiarism detection?	No. Plagiarism detection matches text against a corpus of existing documents; AI detection estimates a probability from statistical patterns with no source to point to.
How accurate are AI detectors?	It depends heavily on the text. Detection is most reliable on clean, unedited, long-form AI output and degrades sharply on short, edited, paraphrased, or non-native English text.
What is the biggest documented failure?	Bias against non-native English writers. A 2023 Patterns study found detectors flagged over 61% of human-written TOEFL essays as AI-generated.
Can a detector prove someone used AI?	No. It produces a probability estimate, not evidence. It should never be the sole basis for an academic or employment decision.
Minimum text length for a usable result?	Around 50 words. Below that, there isn't enough statistical signal for any tool to be reliable.

Frequently asked questions

Can AI detectors be wrong?

Routinely, in both directions. They produce false positives (flagging human writing as AI) and false negatives (missing AI writing, especially if it's been edited). The best-documented false positive pattern is against non-native English speakers, where a 2023 study in Patterns found detectors misclassified more than 61% of human-written TOEFL essays as machine-generated.

Why do different detectors give different scores for the same text?

Because they use different signal weightings, different training data, and different thresholds. One tool may weight perplexity heavily while another leans on model-specific fingerprints. There's no shared standard, no shared benchmark, and no regulator — so disagreement between tools is normal and doesn't mean one of them is broken.

Can teachers prove a student used AI with a detector?

No. A detector output is a probability derived from statistical patterns, not evidence of authorship. Several universities — Vanderbilt among the earliest, in August 2023 — have disabled AI detection tools partly for this reason. A detector score can reasonably start a conversation about a student's drafting process; it cannot substitute for one.

Does running text through a paraphraser defeat AI detection?

Often, yes — which is a real limitation rather than a feature we'd defend. Paraphrasers raise perplexity and burstiness, the two properties most detectors depend on. Some residual patterns survive and can be screened for, but confidence drops meaningfully on heavily rewritten text, and any tool claiming otherwise is overstating what it can do.

Is there a minimum length for reliable AI detection?

Around 50 words as a practical floor, and longer is better. Perplexity and burstiness are measures of distribution across a passage — a couple of sentences don't provide enough of one to be meaningful.

Sources

Liang, W., Yuksekgonul, M., et al. (2023). GPT detectors are biased against non-native English writers. Patterns (Cell Press), Article 100779.
Vanderbilt University Brightspace (2023). Guidance on AI Detection and Why We're Disabling Turnitin's AI Detector.
Pangram Labs. How accurate is Pangram AI detection on ESL? (published April 2025, updated May 2026).

Go deeper

On accuracy and the research:

On academic integrity:

Detecting specific models:

ChatGPT · Claude · Gemini

Choosing a tool:

GPTZero · Originality.ai · Grammarly · ZeroGPT · Turnitin alternatives · 7 free no-signup detectors tested

By use case:

SEO agencies · resumes and cover letters

If you want to try the signals described here on your own text, the free AI detector runs unlimited scans with no account and returns the sentence-level heatmap rather than a single number. How it works and what it can't do are documented on the about page.