Can you really tell when writing was made by AI, or do you only feel sure you can?

This explores what signals lead readers, reviewers or evaluators to suspect that a piece of writing they're judging was produced with AI. The corpus doesn't have research on reviewers of academic papers. What it does have is evidence from neighboring settings, and it points somewhere unexpected: the cues readers rely on are often weaker than they think, and AI writing leaves its clearest traces in places readers don't usually look.

Start with the setting closest to academic evaluation. Admissions officers in one experiment could often tell AI-written essays from human ones, and they rated essays they believed were AI-generated lower Do admissions officers penalize essays they suspect are AI-written?. But elsewhere, the confidence behind an accusation doesn't match the evidence. When researchers looked at online comments that people had accused of being AI-written, those comments lacked the features that actually separate AI text from human text. That suggests the accusations worked more as gatekeeping than detection, and the people harmed were the real human writers who weren't believed Do unfounded AI accusations harm human writers instead?. In another study, readers simply couldn't tell AI-assisted writing from writing done alone Do readers value writing authenticity they cannot detect?.

So what does AI writing actually change? One large study (2,939 writers, 11,091 readers) found that AI assistance shifted how readers saw the writer on all 29 traits measured. The writers came across as more confident, more extreme, more agreeable, and higher in quality Does AI writing assistance change how readers perceive the writer?. They also came across as more privileged: more educated, higher-income, and more likely to be native English speakers than they really were. Researchers call this 'identity laundering': a distinctive personal voice gets flattened into a generic, polished one Does AI writing make authors seem more privileged than they are?. And these shifts reach readers almost untouched, because writers edited AI-generated paragraphs only 23% of the time, and even then barely changed them Do writers actually edit AI-generated text before publishing?. None of these studies tested whether that polished, self-assured voice is what makes readers suspicious. But it's a reasonable guess about the 'feel' that evaluators react to. It also suggests a risk: a writer whose natural voice is unusually fluent or formal could be suspected for the same reasons.

The strongest detection evidence comes from structure, not wording. In fiction, a system called StoryScope told AI stories from human ones with 93% accuracy using only narrative choices like how characters act and how events are ordered. It kept nearly all of that accuracy after every stylistic cue was removed Can AI stories be detected without analyzing writing style?. For academic writing, the parallel would be how an argument is built, not which words appear in it. That's a guess, though: the corpus hasn't tested it on papers. Whether heavy rewriting also fools automated detectors is still an open question Do rewrites that hide authorship also fool AI detectors?.

Finally, suspicion matters partly because of what happens after it. When readers are told AI was used, trust drops, most sharply in personal writing How does revealing AI authorship change reader trust?. The penalty on a straightforward news article was small but consistent, and it showed up even when the raters were LLMs Does disclosing AI assistance make readers trust articles less?. Readers with more AI literacy penalize less Does AI literacy reduce the damage from AI disclosure?. And readers think disclosure is more necessary than writers do, especially when AI text is pasted in directly and couldn't easily be replaced Do readers and writers differ on AI disclosure necessity?. The takeaway: what readers suspect, and how harshly they judge it, depends as much on their expectations as on anything in the text.