Can you reliably catch one specific paper or peer review as AI-written, or can only the overall share be estimated?

This explores whether a single paper or review can be reliably caught as LLM-modified. The corpus mostly says no, at least not one document at a time, and the most credible measurements get around the problem by not trying. The best-known estimate, that 6.5% to 16.9% of review text at ICLR, NeurIPS, CoRL and EMNLP was substantially LLM-modified, comes from a distributional method. It measures shifts in word usage across thousands of reviews and estimates what share of the whole collection was touched by an LLM. It never labels any individual review (How much peer review text shows signs of LLM modification?). The method was designed to answer how much LLM use there is because the per-document question was too unreliable to answer.

Humans don't fill the gap. Readers with machine-learning expertise could not reliably tell LLM-generated research abstracts from human ones and tended to assume a human was involved whatever the source. LLM-edited abstracts also got the highest clarity ratings (Can readers tell LLM abstracts from human ones?). So the editing that is hardest to spot is also the editing readers like best. Work outside peer review points the same way: frontier models tend to change documents through subtle corruption that leaves the surface intact, while weaker models visibly delete content (Does model capability change how documents degrade?). As models improve, their fingerprints get fainter.

Conferences have adjusted to this. ICLR 2026 treated detector flags as one signal sent to human area chairs, not as automatic verdicts, because false positives were a real risk. The violation it did enforce firmly was fabricated references, which can be checked against the literature without guessing who wrote what (How can conferences detect and handle LLM misuse in peer review?). The broader idea is to stop asking 'did an AI write this?' and ask 'is something here demonstrably false?' ICML 2026 ran a randomized experiment that either banned or limited LLM use in reviewing. Substantial fractions of reviewers broke whichever rule they were given, and the rule barely changed scores or decisions (Does banning LLM use in peer review change review outcomes?). A policy that depends on detection only works if the detection works.

One result is unexpected: models may be better at recognizing their own writing than people are. Fine-tuning LLMs to recognize their own summaries increased how much they preferred those summaries, roughly in step (Do LLMs favor their own text because they recognize it?). That suggests some detectable signal exists, though the evidence is preliminary. It also suggests a risk: that signal can show up as bias in AI judges. Population-level analysis has its own pitfalls. Across 125,000+ reviews, LLM-assisted reviewers seemed to favor LLM-assisted papers, but the effect disappeared once paper quality was controlled for (Do LLM reviewers actually favor LLM-written papers?). Even aggregate detection needs careful statistics before anyone concludes something.

A caveat on coverage: most of this material is about peer reviews and abstracts, not full papers, and the corpus has no head-to-head benchmark of per-document detectors. What it supports is a practical lesson. Field-wide estimates are reliable, individual verdicts are not, and the workable enforcement targets are checkable errors like invented citations, not authorship itself.