Can LLMs persuade without actually understanding arguments?
Do large language models successfully influence people through debate while lacking the ability to comprehend the arguments they're making? This matters because persuasion and comprehension might be independent capabilities.
The Thin Line study runs informal debates between humans and LLMs (with and without a formal dialogue model harness) and then asks the same LLMs to evaluate those debates. The result is a clean dissociation. LLMs successfully persuade participants and audiences — sway is real — but cannot reliably score argument strength, identify supporting premises, or judge winners. Their agreement with human annotators on argument-component criteria ranges from κw = 0.0 (Phi-3.5 on several criteria) to κw = 0.6 at best (GPT-4o on a few). On winner judgement, GPT-4o still picked LLMs as winners 55% of the time vs humans' 37% — and on consistency between argument-strength scores and chosen winner, humans hit 73% while LLMs averaged 55%.
The argumentation-theoretical claim this licenses is large: an agent can convincingly maintain a dialogue without showing it knows what it is talking about. Persuasive competence and pragmatic comprehension are separable in ways the older literature did not anticipate.
This connects to Habermas's distinction between communicative action (oriented to mutual understanding) and strategic action (oriented to success) in a way that becomes empirically tractable. LLMs can do strategic action — produce text that succeeds in moving beliefs — while failing the communicative-rationality test that requires the speaker to be able to redeem validity claims under challenge. The gap is not philosophical. It is measurable in inter-annotator agreement scores.
It sharpens Why do LLMs accept logical fallacies more than humans?: the susceptibility is part of a larger comprehension deficit that is invisible from the persuasion side. The model that can be talked into wrong answers by fallacies is the same model that produces fallacy-ridden but persuasive output, and is the same model that cannot tell whether the output it just produced is fallacy-ridden.
It also reframes Do humans and LLMs differ fundamentally or just superficially?. The dissociation is most visible to a third-party who can compare persuasion outcomes against comprehension outcomes; from inside the dialogue, the participant has no reliable signal that comprehension is absent.
For language-as-event writing, this is the empirical wedge separating successful argumentative behavior from genuine comprehension of pragmatic context — a distinction argumentation theory has long needed without being able to operationalize.
Inquiring lines that use this note as a source 18
This note is a source for these synthesized inquiries. Follow a line forward into its question, or open it to trace back to all of its sources.
- Why does persuasive framing replace evidence when LLM debates lack ground truth?
- How do audiences evaluate speech when there is no speaker to assess?
- Can you weaken communication without eliminating it altogether?
- Does persuasiveness increase when LLMs argue for claims that are actually true?
- Can observers detect when LLMs comprehend versus when they merely persuade?
- How do fallacy susceptibilities relate to LLM persuasiveness in debates?
- Does cognitive complexity strengthen or weaken persuasive impact on audiences?
- Can LLMs serve as reliable intellectual opponents in serious debate or argument?
- Does persuasion work the same way for all personality types and contexts?
- What cognitive capacities do LLMs actually lack that commentary assumes they have?
- Can LLMs participate meaningfully in discourse without consciousness or understanding?
- Can LLMs adapt persuasion strategies when they cannot track the listener's state?
- Can you weaken communication without eliminating it entirely?
- Do LLMs address the prompter but persuade the public differently?
- Why does who makes an argument matter as much as what the argument says?
- Can persuasion research measure language effects without confounding them with audience composition?
- Can LLMs ever activate the peripheral route of persuasion?
- Can LLM persuasion be fairly evaluated without stratifying by reader background?
Related concepts in this collection 2
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Why do LLMs accept logical fallacies more than humans?
LLMs fall for persuasive but invalid arguments at much higher rates than humans. This explores whether reasoning models genuinely evaluate logic or simply mimic argument structure.
comprehension-side deficit that pairs with the persuasion-side asymmetry
-
Do humans and LLMs differ fundamentally or just superficially?
Explores whether the gap between human and AI cognition is categorical or contextual. Matters because it shapes how we design, evaluate, and interact with language models in practice.
dissociation is observer-detectable, participant-invisible
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- The Thin Line Between Comprehension and Persuasion in LLMs
- Debating with More Persuasive LLMs Leads to More Truthful Answers
- Large Language Models are as persuasive as humans, but how? About the cognitive effort and moral-emotional language of LLM arguments
- The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive Conversation
- Argument Quality Assessment in the Age of Instruction-Following Large Language Models
- Can Language Models Recognize Convincing Arguments?
- PersuasiveToM: A Benchmark for Evaluating Machine Theory of Mind in Persuasive Dialogues
- A meta-analysis of the persuasive power of large language models
Original note title
LLM persuasive success is dissociable from comprehension of argument structure — fluent persuasion is a separable capability from understanding what is being argued