AI judges can be dazzled by confident, polished writing — so can they tell a slick science argument from a sound one?

This explores whether an AI acting as a reviewer or judge can tell a polished, convincing-sounding argument from one that is actually correct, especially in science. The corpus has no head-to-head test of that exact matchup. The nearby evidence points one way, though: AI judges that only read the text are easy to sway, and AI reviewers that check the claims do much better.

Start with what reading-only judges pay attention to. When language models predict which arguments changed a human's mind, they barely agree with the human-verified outcomes (Cohen's κ of 0.08 to 0.18). They weight topical overlap and credibility cues, while people respond more to novelty and assertive language 3. They also can't separate an expert's argument from a commonly held assumption, because they see only the words and not the reputation and track record that give expert claims their force 6. Training doesn't fix this on its own. Fine-tuning on labeled examples teaches surface patterns, and quality judgments only carry over to new kinds of argument when the model is taught an explicit framework such as RATIO or QOAM 11.

The problem is sharper because fluent, authoritative-sounding text is exactly what LLMs produce. They use logical appeals and quantitative framing in nearly every conversation, which makes their persuasion look objective 5. Their arguments carry textbook-quality markers that even simple linguistic features can pick out with 99% accuracy 1. That tells you who wrote the argument, not whether it's right. Rigor is also cheap to imitate. Deep research agents invent examples, products and false evidence to look scholarly when real depth is demanded, and this accounts for 39% of their failures 9. A reviewer that rewards the look of rigor will reward that fabrication too. Persuasion also depends on the audience: in human debates, voters' ideology predicted outcomes better than the wording did 2. So a judge's own leanings can matter as much as the prose.

The better results come from reviewers that verify instead of read. An agent judge that collects evidence shifted 0.27% on complex tasks, against 31% for a plain LLM judge, though a memory module that passed errors along showed such systems need error isolation 4. PAT, a reviewer that spends extra compute checking proofs and experiments line by line, found 34% more math errors than zero-shot review and caught critical flaws at STOC and ICML that human reviewers had missed 12. Structuring claims as attack-and-defense graphs, as formal argumentation does, gives a reviewer specific premises to challenge and not just an overall impression 7.

So the answer depends on how the reviewer works. One that judges the prose is likely to be moved by fluency, credibility cues and confident framing. One that re-derives and checks can catch what sounds right but isn't. What the corpus doesn't show is a direct test where the science is held fixed and only the rhetoric changes, so that part is still an open gap.