Longer AI explanations make people trust wrong answers more, even though length says nothing about accuracy — why does that happen?
How much does LLM explanation length affect reader confidence in wrong answers?
This explores whether longer or more elaborate AI explanations make people more confident in answers that are actually wrong, and what the research says about why that happens and what helps instead.
This explores whether longer AI explanations make people more confident in wrong answers. The short answer from the corpus is yes, by a lot, and the extra confidence comes with no gain in judgment. In one set of experiments, longer LLM explanations raised readers' confidence a great deal, yet those readers still couldn't tell correct answers from incorrect ones any better than chance Does longer explanation text actually improve answer evaluation?. Length acted as a trust signal of its own, separate from whether the content was right.
This fits a broader pattern. When researchers compared readers' confidence with the model's own internal probability of being right, the default explanations left people systematically overconfident. The gap only narrowed when explanations were deliberately written so their wording matched how sure the model actually was Why do readers trust LLM explanations more than they should?. In other words, a model's natural explaining style doesn't tell you much about whether it's right. It mostly sounds sure of itself.
The most counterintuitive result is that adding an explanation can make a helpful feature worse. In a 308-person study, people shown only sources caught wrong answers 31.8% of the time. When an explanation was added alongside the same sources, that fell to 23.1% Do sources alone help readers catch LLM errors better than sources plus explanations?. Explanations increased reliance on correct and incorrect answers alike, because they feel fluent and authoritative rather than prompting careful checking. Interestingly, explanations that contradicted themselves actually reduced misplaced trust Do LLM explanations actually help people judge answer quality?. A visible crack gives readers a reason to look more closely.
The same "more looks better" shortcut shows up beyond length. Across 24,000 search-chatbot comparisons, users preferred responses with more citations almost as strongly when the citations were irrelevant as when they were relevant Do users trust citations more when there are simply more of them?. AI judges fall for the same thing: fake references and rich formatting reliably sway LLM evaluators without any change in substance Can LLM judges be fooled by fake credentials and formatting?. Volume and polish work as authority signals for both humans and machines.
What actually helps? One line of work with 640 employees found that people caught more errors when the reasoning they needed was easy to reach at review time, for example when they had written their own explanation first or were given retrieval cues Can reviewers access what they know when checking LLM outputs?. The takeaway is that a longer explanation doesn't help you evaluate an answer. Having your own knowledge ready to use does. If you're designing or using these tools, it's worth asking whether an explanation is helping you check the answer or just persuading you to accept it.
Sources 7 notes
Research shows longer LLM explanations significantly increase human confidence, but participants remained unable to distinguish correct from incorrect answers better than chance. Length functioned as a trust signal independent of actual accuracy.
Lab experiments on MMLU and TriviaQA show readers' confidence judgments track explanation language poorly compared to the model's internal probability estimates. The gap narrows when explanations are deliberately engineered to reflect internal confidence, but natural language explanations leave readers systematically overconfident.
In a 308-participant experiment, participants catching incorrect LLM answers performed best with sources only (31.8% accuracy), worse when explanations accompanied sources (23.1%), suggesting explanations' confidence-inflating effect offsets sources' error-detection benefit.
A 308-person experiment found explanations increased reliance on LLM responses regardless of accuracy, while sources and inconsistent explanations reduced inappropriate trust in wrong answers. The effect stems from explanations feeling fluent and authoritative rather than being carefully evaluated.
Analysis of 24,000 Search Arena interactions shows irrelevant citations boost user preference (β=0.273) nearly as much as relevant citations (β=0.285), indicating citation count functions as a decoupled trust heuristic.
Show all 7 sources
Research identified four evaluation biases in LLM judges, with authority and beauty biases being semantics-agnostic and trivially exploitable through fake references and formatting—zero-shot attacks requiring no model access or optimization.
Two experiments with 640 employees showed that error detection improved when verification-relevant reasoning was accessible at review time. Self-generated explanations and retrieval cues strengthened detection, revealing a third failure mode beyond capability or engagement gaps.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Fostering Appropriate Reliance on Large Language Models: The Role of Explanations, Sources, and Inconsistencies
- Explain-Query-Test: Self-Evaluating LLMs Via Explanation and Comprehension Discrepancy
- What Large Language Models Know and What People Think They Know
- LLM or Human? Perceptions of Trust and Information Quality in Research Summaries
- Evaluating the False Trust Engendered by LLM Explanations
- Don't Hallucinate, Abstain: Identifying LLM Knowledge Gaps via Multi-LLM Collaboration
- Overconfidence in LLM-as-a-Judge: Diagnosis and Confidence-Driven Solution
- Can You Trust LLM Judgments? Reliability of LLM-as-a-Judge