Catching AI-written papers at scale is unreliable, so the stronger defense may be checking whether their claims are true.

This explores whether there are dependable ways to catch AI-written research papers when thousands are submitted at once. The short answer from this collection is that no method here reliably detects AI writing in papers at scale. The more useful finding is that the best defenses stop asking who wrote the text and start checking what the text claims. People are a weak first line. Readers with machine-learning expertise couldn't reliably tell LLM-written abstracts from human ones and tended to assume a human was involved. Abstracts that an LLM had edited got the highest clarity ratings and were preferred 55% of the time Can readers tell LLM abstracts from human ones?. So most AI involvement in papers doesn't look like a fake. It looks like a polished human paper.

Automated stylistic detection does work in some settings. Simple, readable language features (the way an LLM goes along with its prompt, its 'textbook-quality' argument markers) caught AI-written counter-arguments on Reddit with 99% accuracy, matching heavy neural detectors at a fraction of the cost Can simple linguistic features detect AI-written arguments?. That's promising, but it's a different genre. Short, informal Reddit arguments carry stronger style signals than formal academic prose, which already sounds a bit like a textbook. The corpus has no comparable result for full papers, so treat that 99% as a lead worth following, not a solution.

The clearest real-world example is ICLR 2026. Its program chairs treated detector flags as one input passed to human area chairs, not as automatic rejections, because the detectors produce false positives. The one thing they enforced hard was fabricated references: papers with confirmed made-up citations were desk-rejected How can conferences detect and handle LLM misuse in peer review?. That's the key move. A fake citation can be checked against a database at scale, and it's wrong whoever wrote the paper. Writing style can't be checked that way.

The threat itself suggests that style detection is aimed at the wrong thing. One demonstration produced 288 complete finance papers from 96 statistically significant patterns found in the data, each with an invented theory and made-up citations Can AI generate hundreds of fake academic papers automatically?. The problem there isn't that a machine wrote the prose. It's that the theories were invented after the results were known (a practice called HARKing), and AI made that cheap. Capability makes things worse: weaker models visibly drop content, while frontier models corrupt documents in subtle ways that keep the surface looking intact Does model capability change how documents degrade?. So the better the generator, the less there is on the surface to detect.

One trap to know about: having an LLM screen papers creates a weakness of its own. LLM judges score text higher when it has fake references and rich formatting, regardless of content Can LLM judges be fooled by fake credentials and formatting? Can LLM judges be tricked without accessing their internals?. People show the same bias: irrelevant citations raise trust almost as much as relevant ones Do users trust citations more when there are simply more of them?. A fabricated paper is built to exploit exactly these signals, which is another reason to verify citations directly instead of trusting how authoritative a paper looks.