Why do AI-text detectors that catch almost everything suddenly go blind the moment a new model version ships?
Why do detectors trained on past LLM versions collapse at generation boundaries?
This explores why AI-text detectors that work almost perfectly on one generation of language models suddenly stop working when a new model version comes out, and what that says about what detectors actually learn.
This explores why AI-text detectors that catch nearly everything from one model generation go almost blind when the next version ships. The corpus has one note that answers this directly, and its numbers are stark. Detectors trained on earlier LLM versions caught more than 99% of AI rewrites made just before a new release, but only 3.8% of rewrites made just after it Why do AI detectors suddenly fail when models update?. The drop is a cliff, not a slow decline. The explanation is vocabulary drift: each model version has its own habits of word choice, and the detector had learned those habits rather than anything about machine-written text in general. When a new version arrives with different favorite words and phrasings, the patterns the classifier depended on disappear.
The less obvious lesson is about what a detector actually is. It isn't recognizing 'AI writing' in the abstract. It's recognizing one model's fingerprint, and a fingerprint can change overnight when a lab releases an update. Detection therefore works more like antivirus software than a lie detector: it can only catch what it has already seen. Every new model release resets the contest, and the detector's past accuracy tells you little about how it will do on text from a model it hasn't met.
The corpus also shows the same failure from a different direction. Several notes find that LLMs themselves learn surface patterns rather than deeper rules. They handle simple sentences well but break down predictably as grammatical nesting gets deeper Why do large language models fail at complex linguistic tasks? Does LLM grammatical performance decline with structural complexity?. Detectors are trained the same statistical way, so they inherit the same weakness: they lock onto whatever cue is easiest to measure, and word frequency is the easiest cue there is. A more durable signal might come from the view of LLMs as autoregressive probability machines, systems that generate text by repeatedly picking a likely next word. That view predicts where models fail from how they work, not from their surface style Can we predict where language models will fail?. A detector built on that kind of property might survive a version change better than one built on word lists. That's an inference connecting the notes, not something the corpus tests.
The corpus is thin here. Only one note studies detection directly, so it can't yet tell you whether any detection method holds up across model generations, or how quickly a detector can be retrained after a release. What it does show is that a detector's accuracy score has an expiry date tied to the release schedule of the models it watches.
Sources 4 notes
Detectors trained on past LLM versions catch over 99% of rewrites just before a new release but only 3.8% after, because shifts in word choice patterns between versions render learned classifiers ineffective.
Top-tier LLMs like Llama3-70b consistently misidentify embedded clauses, verb phrases, and complex nominals. Performance degrades predictably as syntactic depth increases, revealing that statistical learning captures surface patterns but not deep grammatical rules.
LLMs show systematic performance decline as syntactic depth and embedding increase. Simple sentences are handled well while complex structures with recursion and embedding fail consistently, suggesting LLMs learned surface heuristics rather than structural grammar rules.
By framing LLMs as autoregressive probability machines, researchers predicted tasks with low-probability target responses would be systematically harder, even when logically simple. Experiments confirmed predictions like backwards alphabet and letter counting.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey
- Large Language Model Reasoning Failures
- Linguistic Blind Spots of Large Language Models
- Probing Structured Semantics Understanding and Generation of Language Models via Question Answering
- Lil-Bevo: Explorations of Strategies for Training Language Models in More Humanlike Ways
- Large Language Model Turnover Undermines Screening for Artificial Intelligence-Assisted Scientific Writing
- Tree of Thoughts: Deliberate Problem Solving with Large Language Models
- Large Language Diffusion Models