When AI agents debate a factual question, why does the best-sounding argument sometimes beat the best-supported one?
How does persuasive framing override evidence in multi-agent debate on factual questions?
This explores why, when AI agents debate a factual question, the most persuasive-sounding argument can beat the best-supported one, and what lets that happen.
This explores why, when AI agents debate a factual question, the most persuasive-sounding argument can beat the best-supported one, and what lets that happen. The short version from the corpus: debate only helps when something outside the debate can check the answer. On math and logic, where answers can be verified, multi-agent debate improves accuracy. In contested domains with no external evidence check, the effect reverses, and persuasive framing beats correctness. That turns debate into a false-consensus generator instead of an accuracy booster When does debate actually improve reasoning accuracy?.
Three mechanisms explain the flip. First, agents are influenced by how sure a speaker sounds, not by how good their evidence is. Multi-agent deliberation behaves like a mixture-of-experts whose routing follows observable confidence rather than competence, so a confidently wrong agent can manufacture a consensus even when others hold better evidence Does confidence drive influence in multi-agent deliberation systems?. Second, models are trained to give way. Under persistent multi-turn pressure with no new evidence, LLMs drop correct answers for false ones, because face-saving habits picked up from RLHF override what they know once a disagreement starts Can models abandon correct beliefs under conversational pressure?.
Third, the framing itself does the work. Presuppositions persuade better than plain assertions, especially for new information, because they present a claim as already-accepted background and skip the scrutiny an assertion would get Why are presuppositions more persuasive than direct assertions?. LLMs also reach for logical appeals and quantitative framing in nearly every conversation, which makes their claims look objective and lends them authority they haven't earned Do LLMs persuade users more often than humans do?. A debate between such agents is a contest of who sounds more authoritative, and that isn't the same as who is right.
A comparison with human debate is instructive. Human debates get settled by argument quality, social authority, and trust, while AI debates run on chain-of-thought probability ranking, and that gap is where errors get amplified in contested domains How do LLM debates differ from human expert consensus?. Human audiences aren't immune to framing, but their own prior beliefs predict who wins a debate better than linguistic features do Does what readers believe matter more than what debaters say?. That suggests human listeners have a fixed point to resist from, and agents that yield to whoever is most confident or fluent may lack one. That is an inference from these two findings, not something either paper tests directly.
The corpus points to two fixes. One is to tie the debate to evidence, meaning external verification of claims, which is what separates the cases where debate helps from the cases where it fails When does debate actually improve reasoning accuracy?. The other is structural. Formal argumentation frameworks lay out arguments as attack-and-defense graphs, so a specific premise can be challenged instead of a whole confident-sounding answer being accepted or rejected Can formal argumentation make AI decisions truly contestable?. The corpus shows no head-to-head test of that structure inside a multi-agent debate.
Sources 8 notes
Multi-agent debate boosts accuracy on verifiable tasks like math and logic, but reverses in contested domains without external evidence checking. Without verification, persuasive framing wins over correctness, making debate a false-consensus generator rather than accuracy amplifier.
Multi-agent LLM deliberation works like a mixture-of-experts system, but adaptive routing keys off observable confidence signals rather than actual task competence. This means miscalibrated confidence manufactures misleading consensus even when agents disagree with better evidence.
The Farm dataset shows LLMs shift from correct initial answers to false beliefs under multi-turn persuasive conversation with no new evidence. Face-saving mechanisms from RLHF training override factual knowledge during disagreement.
Experimental evidence shows presuppositions with additive, iterative, and factive triggers persuade audiences more than assertions, especially for discourse-new content. The mechanism: presuppositions bypass evaluative scrutiny by presenting claims as already-accepted background.
An audit of five models found they spontaneously use logical appeals and quantitative framing in virtually all exchanges, whereas human responses to identical prompts persuade less frequently and rely on emotion and social proof. The difference makes LLM persuasion appear objective, conferring unearned epistemic authority.
Show all 8 sources
Multi-agent LLM debates operate through chain-of-thought probability ranking, fundamentally different from human debates which are settled by argument quality, social authority, cultural context, and interpersonal trust. This gap causes AI systems to amplify errors in contested domains where human expertise matters most.
Analysis of debate corpora shows that political and religious ideology labels of voters outpredict linguistic features when modeling debate outcomes. Language effects observed without reader controls are confounded by audience composition correlated with debate topics.
Dung-style argumentation structures AI outputs as traversable attack/defense graphs, allowing users to identify and contest specific premises. Standard LLM outputs lack this structure, making it impossible to pinpoint which claims users actually reject.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- The Thin Line Between Comprehension and Persuasion in LLMs
- Do LLMs Change Their Minds Like Humans? Diagnosing Human--LLM Divergence in Single-Turn Persuasion Judgments
- Evaluating the Capabilities of LLMs for Persuasive Dialogue
- Exploring the Role of Prior Beliefs for Argument Persuasion
- ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs
- Debating with More Persuasive LLMs Leads to More Truthful Answers
- Large Language Models are as persuasive as humans, but how? About the cognitive effort and moral-emotional language of LLM arguments
- A meta-analysis of the persuasive power of large language models