Can one detector catch all LLM versions at once?

Beyond showing that single detectors collapse at model-generation boundaries (see Why do AI detectors suddenly fail when models update?), the authors tested two ways an institution could cover all 23 LLM versions in its training data at once, and found neither escapes the trade-off. In the "union" scenario, 23 version-specific detectors each run at their own 1%-false-positive threshold, and a manuscript is flagged if any one fires; this "catches nearly every AI-rewritten abstract" — 98.6% even for the hardest version — but "flags roughly one in eight genuinely human abstracts" (11.9% ± 1.2%), because the false flags of 23 separate detectors only partly overlap and so accumulate. In the "pooled" scenario, a single detector trains on rewrites from all 23 versions together; it "rarely flags human-written abstracts" (0.8% ± 0.3%, close to its 1% target) but still misses a median of 2.8% of rewrites across versions, concentrating on the newest ones — 33.8% ± 3.4% missed for the hardest version (called Muse-Glimmer), even though that version's rewrites are present in its own training data.

The mechanism is accumulation versus dilution. Union accumulates false positives because every added per-version detector contributes its own 1% noise against the human-written set, and those errors are not the same abstracts across detectors. Pooling dilutes a problem instead of accumulating it: training one detector on a mix of versions keeps the false-positive rate anchored near its target, but the hardest version's distinguishing signal gets averaged away among the other 22 versions' training examples, so detection on that version degrades even with direct exposure to its own rewrites in training.

This result gives the version-boundary-collapse finding above an operational face: it is not just that a maintenance lag causes failure, it is that keeping pace with every released version still leaves a real choice between over-flagging authors and under-catching undisclosed AI use, directly analogous to the asymmetric costs the authors describe up front — a wrongly flagged author must contest a rejection, while an undisclosed AI-written manuscript that passes puts the burden on reviewers and editors. It extends the caution already implicit in How can conferences detect and handle LLM misuse in peer review?: routing flags to a human area chair is one way to absorb exactly the 1-in-8 false-flag cost that a union-style screen would otherwise impose directly on authors.

The excerpt measures this trade-off only for PNAS-style scientific abstracts, one architecture (fine-tuned RoBERTa-base), and 23 versions from three vendors over roughly three years; it does not establish the trade-off's exact magnitude for other document types, detector architectures, or future model releases, nor does it say which point on the trade-off a given journal or conference should choose. What it does establish is that "covering all versions" is not itself a solution — the choice of union versus pooled training determines which side of the research community (authors or reviewers) absorbs the residual error, and that choice has to be made deliberately rather than assumed away by broader training data.