When two AIs debate and a judge picks a winner, does the best argument win, or just the most persuasive one?
Why does inference-time debate fail when persuasion substitutes for evidence?
This explores why setting AI models against each other to argue a question, then letting a judge pick the winner, can reward the best-sounding case over the best-supported one.
This explores why AI-versus-AI debate at inference time can reward the best-sounding case instead of the best-supported one. The corpus has no note that tests debate directly, so what follows is assembled from what it knows about machine persuasion, how models produce arguments, and how judges get swayed. Three weaknesses stack up.
The first is the arguer. Persuasiveness in LLMs doesn't seem to depend on being right. Claude beats incentivized humans at both truthful and deceptive persuasion, and DeepSeek beats them only when arguing for falsehoods, which suggests the mechanism is content-independent Do large language models persuade better than humans?. Models also reach for logical appeals and quantitative framing in nearly every conversation, even when it isn't warranted. That makes them look objective and lends them authority they haven't earned Do LLMs persuade users more often than humans do?. If both debaters always sound rigorous, the judge gets no signal about which side actually has the evidence.
The second is the form of the reasoning itself. Chain-of-thought exemplars that are logically invalid perform nearly as well as valid ones, so the gains come from the shape of reasoning, not its validity Does logical validity actually drive chain-of-thought gains?. Chain-of-thought looks more like constrained imitation than inference, and structural coherence matters more than correctness Why does chain-of-thought reasoning fail in predictable ways?. A model can therefore produce a debate-shaped argument (premise, statistic, conclusion) with little underneath. Presupposition makes this worse. Claims framed as already-accepted background persuade more than direct assertions, especially for new information, because they skip scrutiny Why are presuppositions more persuasive than direct assertions?. A judge that only checks the claims debaters assert never examines the ones slipped in as background.
The third is the judge. In human debate corpora, voters' political and religious ideology predicts who wins better than the debaters' language does Does what readers believe matter more than what debaters say?. Nobody in this corpus tests whether model judges behave the same way. If they do, debate would mostly echo the judge's existing leanings instead of settling the question. The timing matters too. AI persuasive advantage was strong at first contact and eroded over repeated rounds with the same person Does AI persuasiveness fade across repeated conversations with the same person?. A one-shot debate verdict catches the effect near its peak.
The closest thing to a remedy is faithfulness training. Small models can learn to ground answers in a passage, cite literal quotes, and abstain instead of confabulating, and this comes from how the training is designed, not from scale Can small models learn to ground answers in context?. This is an extrapolation, not a tested debate fix. A debate that pays for quotable evidence and lets a debater say "I can't support this" would reward something different from a debate that pays for confident-sounding argument.
Sources 8 notes
Claude beats incentivized humans at both truthful and deceptive persuasion, while DeepSeek only beats them when arguing for falsehoods. The persuasion mechanism appears content-independent, suggesting model family itself acts as a contextual moderator.
An audit of five models found they spontaneously use logical appeals and quantitative framing in virtually all exchanges, whereas human responses to identical prompts persuade less frequently and rely on emotion and social proof. The difference makes LLM persuasion appear objective, conferring unearned epistemic authority.
Illogical chain-of-thought exemplars matched valid CoT performance on BIG-Bench Hard, showing that structural properties—not logical validity—drive the gains. The model learns the form of reasoning, not genuine inference.
CoT guides models to pattern-match reasoning structure rather than perform genuine inference. This explains distribution-bounded failures, why structural coherence matters more than content correctness, and why performance optimizes against interpretability.
Experimental evidence shows presuppositions with additive, iterative, and factive triggers persuade audiences more than assertions, especially for discourse-new content. The mechanism: presuppositions bypass evaluative scrutiny by presenting claims as already-accepted background.
Show all 8 sources
Analysis of debate corpora shows that political and religious ideology labels of voters outpredict linguistic features when modeling debate outcomes. Language effects observed without reader controls are confounded by audience composition correlated with debate topics.
Claude and DeepSeek showed strong initial persuasive advantage, but this edge eroded across repeated quiz rounds while human persuaders maintained consistent effectiveness. This decay pattern is opposite to human-to-human persuasion, where rapport typically strengthens over time.
Sub-2B models trained on synthetic multi-hop QA can ground answers in passages, cite literal quotes, and abstain from confabulation. The OCC-RAG work shows faithfulness emerges from training curriculum design, not parameter count.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Exploring the Role of Prior Beliefs for Argument Persuasion
- Do LLMs Change Their Minds Like Humans? Diagnosing Human--LLM Divergence in Single-Turn Persuasion Judgments
- A meta-analysis of the persuasive power of large language models
- Evaluating the Capabilities of LLMs for Persuasive Dialogue
- The Thin Line Between Comprehension and Persuasion in LLMs
- When Large Language Models are More Persuasive Than Incentivized Humans, and Why
- Large Language Models are as persuasive as humans, but how? About the cognitive effort and moral-emotional language of LLM arguments
- Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens