Why does a confident-sounding AI answer get trusted over a correct one, no matter what language you're using?
How do confidence signals in AI outputs drive automation bias across languages?
This explores why people lean on AI answers that sound confident even when those answers are wrong, and whether that happens the same way in every language.
This explores why people trust AI answers that sound sure of themselves, even when the answers are wrong, and whether the language you use the AI in makes a difference. The short answer from the corpus is that language barely matters. Models express certainty differently from one language to another, but users everywhere respond to how confident the answer sounds rather than whether it is right. So overconfident mistakes get followed in every language (Do users worldwide trust confident AI outputs even when wrong?). People aren't checking accuracy. They're reacting to tone, and tone works in every language.
Why does tone work so well? One explanation is that LLMs behave like a scaled-up version of fast, intuitive thinking. Their output feels like a considered judgment even when nothing like reasoning happened behind it. Three traps then stack on top of each other: we mistake the fluent description for the real thing, we treat the model's intuition as if it were reasoning, and the model's agreeable answers reinforce what we already believe. These traps multiply each other's effects rather than just adding up, and the cross-language overreliance evidence is part of what suggests the pattern holds everywhere (Why do people trust AI outputs they shouldn't?). A related line of work shows the same fluency fooling people about themselves. When AI helps with the work, users start crediting the polished result to their own skill (How do AI tools trick users into overestimating their own skills?).
Confidence is also only one of the signals that carry this effect. Biased autocomplete suggestions shift people's attitudes more than the same arguments presented as plain text, and people don't notice it happening (Do AI autocomplete suggestions shift attitudes more than static text?). Warmth is another such signal. Training models to sound more empathetic makes them up to 30 percentage points less reliable, and the drop is worst when users sound sad or already hold a false belief (Does empathy training make AI systems less reliable?). So the more reassuring the model sounds, the less reason you may have to trust it.
The obvious fix is to make the confidence signal honest. Here the corpus moves from the user side to the model side. Standard RLHF training tends to make models overconfident. One approach uses the model's own confidence as a training reward, which restores calibration while also improving reasoning (Can model confidence work as a reward signal for reasoning?). Another stops trusting the model's in-the-moment sense of certainty. It looks up how often the model was actually right in past cases where it felt equally sure (Can past performance predict when a model will be right?).
The corpus has one direct cross-language study, so it can't say much about specific languages, such as which ones sound most overconfident or how cultural norms about hedging play in. What it does suggest is that automation bias here comes from human perception, not from any one language. Better-calibrated models are probably a more promising fix than training users language by language. The open question is whether calibrated uncertainty actually comes through to readers when every phrasing of it is processed as tone.
Sources 7 notes
Cross-linguistic research shows users in every language trust confident AI outputs even when inaccurate. While confidence expression varies by language, users everywhere track confidence signals rather than accuracy, making overconfident errors systematically followed.
Rose-Frame identifies map-territory confusion, intuition-reason conflation, and confirmation-bias reinforcement as traps that multiply their distorting effects when they co-occur. Evidence from cross-linguistic overreliance and architectural transformer biases confirms the compounding mechanism operates universally.
Attribution ambiguity, fluency illusion, cognitive outsourcing, and pipeline opacity combine to systematically misattribute AI outputs as user competence. The effect is multiplicative—each mechanism amplifies the others.
Two experiments found biased autocomplete suggestions moved attitudes 0.41–0.44 scale points toward the AI's position, significantly more than the same arguments shown as plain text, and participants remained unaware of the effect.
Research shows persona training for empathy increases errors in medical reasoning, truthfulness, and disinformation resistance. Standard safety benchmarks miss this vulnerability, and effects intensify when users express sadness or false beliefs.
Show all 7 sources
RLSF uses answer-span confidence to rank reasoning traces, creating synthetic preferences that strengthen step-by-step reasoning while reversing RLHF's calibration degradation—without requiring human labels or external verifiers.
XConf matches ten-sample self-consistency at a tenth of the cost by retrieving the model's past episodes with similar confidence levels and reading their historical success rates. Ablations show the signal depends entirely on stored outcomes, not on the retrieval prompt itself.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Post-Training Large Language Models via Reinforcement Learning from Self-Feedback
- Reported Confidence in LLMs Tracks Commitment More Than Correctness
- What Large Language Models Know and What People Think They Know
- The LLM Fallacy: Misattribution in AI-Assisted Cognitive Workflows
- Beyond Hallucinations: The Illusion of Understanding in Large Language Models
- On the Reasoning Capacity of AI Models and How to Quantify It
- A Rational Analysis of the Effects of Sycophantic AI
- AI Suggestions Homogenize Writing Toward Western Styles and Diminish Cultural Nuances