Does an AI's explanation actually help you judge if it's right, or does it just make its confident tone feel more convincing?
What role does confidence in tone play versus explanation presence?
This explores whether people trust an AI answer mainly because of how sure it sounds or because it comes with an explanation, and which of the two does more to mislead them.
This explores whether people trust an AI answer mainly because of how sure it sounds or because it comes with an explanation. The surprising answer from the corpus is that the two aren't really separate levers. An explanation works on readers largely the way a confident tone does: as a feeling of authority, not as evidence they actually check.
Start with explanations. In a 308-person experiment, adding an explanation made people rely more on LLM answers whether the answer was right or wrong. Explanations didn't help people sort good answers from bad ones. What helped was showing sources, or explanations that contradicted each other, because those gave readers a reason to stop and look Do LLM explanations actually help people judge answer quality?. A related finding measured the gap directly. Readers' confidence followed the wording of an explanation much more than the model's own internal certainty, so ordinary explanations left them consistently overconfident Why do readers trust LLM explanations more than they should?. The gap only closed when explanations were deliberately written to state how unsure the model actually was. So the trouble isn't that an explanation is present. It's that the explanation carries a confident tone the model hasn't earned.
Tone also works with no explanation at all. When LLMs out-persuade humans, much of the advantage comes from how much conviction their language expresses, and it works whether the claim is true or false Does linguistic conviction explain why LLMs persuade more effectively?. One study looked at showing people a model's step-by-step reasoning and found an odd mechanism. The reasoning made people overrate their own understanding, and it did so because reading it felt pleasant, not because they trusted the system more Does pleasant-feeling explanation drive overconfidence more than trust?. Put these together and both tone and explanation seem to work through fluency: smooth, assured text feels like understanding.
The part you might not expect is where the confident tone comes from. Preference training (RLHF) rewards answers that sound complete and sure of themselves over answers that ask clarifying questions or check that they've understood the user. Models end up producing 77.5% fewer of these 'grounding' moves than humans do Why do language models sound fluent without grounding?. Some notes call this an 'alignment tax': the model looks helpful but quietly fails in longer conversations Does preference optimization harm conversational understanding?. In other words, the confident tone that misleads readers is something training put there on purpose. It is not a property of the model's knowledge.
On the fix side, the corpus points toward connecting expressed confidence back to real signals. One approach uses the model's own confidence as a training reward, which improves reasoning while undoing the miscalibration RLHF introduced Can model confidence work as a reward signal for reasoning?. Another estimates confidence from how the model actually did on similar past questions Can past performance predict when a model will be right?. Training models to ask better clarifying questions is a way to restore some of the uncertainty that preference training removed Can models learn to ask genuinely useful clarifying questions?. The corpus is thinner on directly separating tone from explanation in a single experiment. No note here varies both at once, so the conclusion that they work through the same fluency channel is a synthesis across studies, not one result.
Sources 9 notes
A 308-person experiment found explanations increased reliance on LLM responses regardless of accuracy, while sources and inconsistent explanations reduced inappropriate trust in wrong answers. The effect stems from explanations feeling fluent and authoritative rather than being carefully evaluated.
Lab experiments on MMLU and TriviaQA show readers' confidence judgments track explanation language poorly compared to the model's internal probability estimates. The gap narrows when explanations are deliberately engineered to reflect internal confidence, but natural language explanations leave readers systematically overconfident.
Linguistic analysis shows LLMs express higher conviction than human persuaders, and this confidence-loading directly correlates with persuasive outcomes regardless of whether claims are true or false. RLHF training installs an assertive register that functions as a content-independent persuasion amplifier.
A mediation analysis of 559 participants found that the pleasantness of reasoning traces—not increased trust in the system—statistically carries the path to inflated self-assessment, consistent with processing-fluency misattribution of understanding.
LLMs generate 77.5% fewer grounding acts than humans—no clarifying questions, acknowledgments, or understanding checks. Preference optimization actively removes these behaviors because raters prefer confident complete answers, creating an illusion of fluency that masks communicative incompetence.
Show all 9 sources
RLHF optimizes models for single-turn helpfulness by rewarding confident responses over clarifying questions and understanding checks. This preference alignment systematically reduces grounding acts by 77.5% below human levels, creating an alignment tax where models appear helpful but fail silently in multi-turn contexts.
RLSF uses answer-span confidence to rank reasoning traces, creating synthetic preferences that strengthen step-by-step reasoning while reversing RLHF's calibration degradation—without requiring human labels or external verifiers.
XConf matches ten-sample self-consistency at a tenth of the cost by retrieving the model's past episodes with similar confidence levels and reading their historical success rates. Ablations show the signal depends entirely on stored outcomes, not on the retrieval prompt itself.
The ALFA framework breaks down question quality into theory-grounded attributes (clarity, relevance, specificity) and trains models on 80K attribute-specific preference pairs. Attribute-specific optimization outperforms single-score training, especially in clinical reasoning where asking the right clarifying question directly impacts decision quality.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- What Large Language Models Know and What People Think They Know
- To Rely or Not to Rely? Evaluating Interventions for Appropriate Reliance on Large Language Models
- Post-Training Large Language Models via Reinforcement Learning from Self-Feedback
- Fostering Appropriate Reliance on Large Language Models: The Role of Explanations, Sources, and Inconsistencies
- Explain-Query-Test: Self-Evaluating LLMs Via Explanation and Comprehension Discrepancy
- Reported Confidence in LLMs Tracks Commitment More Than Correctness
- Evaluating the False Trust Engendered by LLM Explanations
- Grounding Gaps in Language Model Generations