Why do AI detectors suddenly fail when models update?

The authors paired 4,000 pre-ChatGPT PNAS abstracts with rewrites by 23 LLM versions from three vendors released between June 2023 and August 2026, then fine-tuned RoBERTa-base detectors under maintenance scenarios ranging from "retrained on every new version" to "trained once and never updated." Calibrated to falsely flag 1% of human-written abstracts, a detector trained only on a vendor's past versions "catch[es] above 99% of rewrites just before the sharpest boundary and 3.8% just after it." The effect is not confined to the lab construction: a commercial detector (Pangram), already deployed for screening, "missed most rewrites of the version just after the sharpest boundary while flagging almost no human-written abstracts" — Pangram's miss rate ranged from 6.5% for GPT-5 to 79.8% for a version the authors call Muse-Glimmer, with no significant correlation to release date (Spearman ρ = 0.25, p = 0.25).

The mechanism the authors give is vocabulary drift, not simple model age: "vocabulary differences between versions largely track where detection transfers and where it fails." A detector trained on a given version's rewrites learns to recognize that version's characteristic word choices; when a new version's output vocabulary shifts enough, the classifier's learned boundary stops applying, even though the new version is chronologically close to the old one and even though the detector's own training set already contained rewrites of that same version pooled in with others (the pooled detector still misses "one in three Muse-Glimmer rewrites even though its training data contain Muse-Glimmer rewrites").

This sits downstream of How can conferences detect and handle LLM misuse in peer review? and Why did AI article share stop growing after 2025?: both describe institutions treating detector output as imperfect, but neither specifies a mechanism for why accuracy is uneven. This paper supplies that mechanism for one case — version turnover — and shows it is not a tail risk: the failure concentrates at a few sharp transitions rather than degrading smoothly, so a detector can look reliable in benchmark testing and then fail almost completely against the very next model release from the same vendor.

The excerpt measures only three vendors' model families and PNAS-style scientific abstracts under one detector architecture (fine-tuned RoBERTa-base) plus one commercial product; it does not establish that all detector architectures degrade at generation boundaries the same way, nor that the same boundaries recur across document types (peer reviews, grant text, other disciplines). The paper's own conclusion is correspondingly narrow but load-bearing for policy: "the benchmark accuracy of a detector [should be treated] as provisional, to be re-verified with every LLM release, including earlier versions" — a one-time benchmark result cannot be read as a standing property of a detector.