Can even peer reviewers reliably spot AI-written research just by reading it, or do they land near chance?

This explores whether people, including peer reviewers, can tell when research writing was produced by AI. The short answer from the collection is: not reliably by eye. The corpus has no study testing human detection on research papers specifically, so the strongest evidence is general. A review of 30 studies covering text, images and voice found that human accuracy generally clusters around chance. It also found that detection has not improved as AI output has become more realistic Can people reliably spot content made by AI?.

The closest thing to a real-world test of research text comes from peer review. Sakana's AI Scientist-v2 submitted three fully AI-generated papers to an ICLR 2025 workshop under double-blind review. One averaged 6.33, which met the acceptance threshold, before it was withdrawn under a protocol agreed in advance Can AI-generated papers pass peer review undetected?. The authors were frank that none of the three met main-conference standards, and they later found a citation error the reviewers had missed Can AI systems generate research papers that pass peer review?. The bar that matters may not be whether a paper sounds AI-written but whether its errors get caught. Related work shows why that matters. Deep research agents invent examples and evidence to seem rigorous Why do deep research agents fabricate scholarly content?. Another demonstration produced 288 finance papers with made-up theoretical justifications and fabricated citations Can AI generate hundreds of fake academic papers automatically?.

Here is the twist: machines often succeed where people fail, because they look at different things. Simple, interpretable linguistic features caught LLM-written arguments on Reddit's r/ChangeMyView with 99% accuracy. They picked up habits like closely mirroring the prompt and textbook-style argument markers Can simple linguistic features detect AI-written arguments?. In fiction, AI stories could be identified at 93% accuracy from narrative choices alone, such as how characters act and how time is ordered. Those signals survive style edits because changing them means rewriting the story, not polishing it Can AI stories be detected without analyzing writing style?. The lesson for research text: the giveaway is probably structural (how arguments are built and evidence is used), not the word choice a human reader would focus on. Whether heavy human rewriting defeats these detectors is still untested Do rewrites that hide authorship also fool AI detectors?.

That matters because most AI text barely gets rewritten. Writers edited AI-suggested paragraphs only 23% of the time, and edited versions stayed about 96% similar to the original Do writers actually edit AI-generated text before publishing?.

The less obvious cost of unreliable human detection falls on human authors. Comments that readers accused of being AI-written lacked the features that actually separate AI from human text. That suggests the accusations work more as gatekeeping than detection, and they can wrongly discredit real people Do unfounded AI accusations harm human writers instead?. Meanwhile, some authors have stopped worrying about human readers altogether. Eighteen arXiv manuscripts hid instructions telling AI reviewers to rate them favourably Are hidden AI prompts in preprints a deceptive research practice?. The detection question now runs both ways: humans trying to spot AI authors, and authors trying to steer AI readers.