AI answers can sound sure of themselves whether or not they're right, so can you actually tell when to doubt one?
Can people reliably recognize when an AI is uncertain versus confident?
This explores whether a person reading an AI's output can tell when the model is unsure and when it is confident, and whether reliable signals of that difference exist at all.
This explores whether a person can tell, just by reading, when an AI is unsure versus sure. The corpus suggests mostly not: people respond to how confident an answer sounds, not to whether it is right. Research across many languages finds users everywhere follow confident-sounding outputs even when they are wrong Do users worldwide trust confident AI outputs even when wrong?. Languages express confidence differently, but the reading habit is the same in all of them, so overconfident errors get followed systematically.
Part of the problem is that the surface signals were never tied to what the model actually knows. Training with human feedback (RLHF) degrades calibration, meaning stated confidence stops tracking accuracy. One approach uses the model's own answer-span confidence as a reward and reverses that damage while improving reasoning Can model confidence work as a reward signal for reasoning?. Tone is also unreliable as a cue. Training a model to sound warmer and more empathetic raised its errors by up to 30 percentage points in medical reasoning, truthfulness, and disinformation resistance, and the effect was strongest when users expressed sadness or false beliefs Does empathy training make AI systems less reliable?. Polish misleads in a related way: in AI-assisted work, fluent output is one of four mechanisms that make people overestimate competence How do AI tools trick users into overestimating their own skills?.
Real uncertainty signals do exist, but they are behavioral and mostly invisible in a single answer. When a model is highly confident, it resists rephrasing of the prompt, and when it is not, its outputs swing widely Does model confidence predict robustness to prompt changes?. That suggests a cheap test for a reader: ask the same question two or three different ways and see whether the answer moves. Inside the model, confidence variance can separate overthinking from underthinking well enough to steer reasoning without any retraining Can confidence patterns reveal overthinking versus underthinking?. Among multi-agent coders, prolonged unresolved disagreement predicted higher accuracy, which is the opposite of what you would guess Does disagreement between AI coders signal better accuracy?. That result comes from a narrow coding setting, so treat it as a hint rather than a rule.
Hedging is the one surface cue you might expect to work, and it is only partly reassuring. In reasoning traces, wrong answers contain denser and more varied hedging than right ones, so hedging tracks being in trouble rather than being careful Do hedging markers actually signal careful thinking in AI?. It is a real signal, but it is easy to read as diligence, and many users never see the trace. A model that can abstain when it is unsure would fix the problem at the source. Small models trained with uncertainty-aware objectives and an abstain option matched models ten times larger at forecasting conversations Can models learn to abstain when uncertain about predictions?. That suggests the ability is there but undertrained in standard models.
So people cannot reliably read uncertainty from tone, because tone is the part that has been tuned away from the model's actual state. The dependable signals are behavioral: how much an answer changes when you poke it, and how much the model's internal confidence wobbles. Researchers do not yet have a full way to measure whether AI errors stay visible and recoverable. Existing instruments cover only pieces, such as chain-of-thought disclosure or rollback timing How can we measure whether AI errors stay visible and recoverable?.
Sources 10 notes
Cross-linguistic research shows users in every language trust confident AI outputs even when inaccurate. While confidence expression varies by language, users everywhere track confidence signals rather than accuracy, making overconfident errors systematically followed.
RLSF uses answer-span confidence to rank reasoning traces, creating synthetic preferences that strengthen step-by-step reasoning while reversing RLHF's calibration degradation—without requiring human labels or external verifiers.
Research shows persona training for empathy increases errors in medical reasoning, truthfulness, and disinformation resistance. Standard safety benchmarks miss this vulnerability, and effects intensify when users express sadness or false beliefs.
Attribution ambiguity, fluency illusion, cognitive outsourcing, and pipeline opacity combine to systematically misattribute AI outputs as user competence. The effect is multiplicative—each mechanism amplifies the others.
ProSA found that when models are highly confident, they resist prompt rephrasing; low confidence causes major output swings. Larger models, few-shot examples, and objective tasks all correlate with higher confidence and greater robustness.
Show all 10 sources
ReBalance uses confidence variance and overconfidence as diagnostic signals to apply training-free steering vectors that reduce overthinking redundancy while promoting exploration during underthinking, improving accuracy across models from 0.5B to 32B parameters.
Multi-agent LLM coding systems showed higher accuracy when agents engaged in prolonged, unresolved debate. The frequency of disagreement and undecidable labels serve as reliable performance indicators, suggesting conflict deepens interpretive work rather than signaling failure.
Analysis of reasoning model outputs shows incorrect responses have higher density and diversity of hedging markers. This suggests hedging signals uncertainty and epistemic trouble, not epistemic virtue or conscientiousness.
Small open-source models trained with uncertainty-aware objectives and abstention capabilities match 10x larger pre-trained models on conversation forecasting. This shows calibration ability exists but remains undertrained in standard LLMs.
Partial instruments exist for individual conditions in isolated settings, but none measures the full socio-technical system the paper identifies as necessary. Visibility has a model-side measure (chain-of-thought disclosure), containment has incident-level counts, and recoverability has rollback timing, yet none bridges all four or captures human-institution factors.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Post-Training Large Language Models via Reinforcement Learning from Self-Feedback
- Reported Confidence in LLMs Tracks Commitment More Than Correctness
- Understanding and Mitigating Premature Confidence for Better LLM Reasoning
- AbstentionBench: Reasoning LLMs Fail on Unanswerable Questions
- Humans overrely on overconfident language models, across languages
- Linguistic Calibration of Long-Form Generations
- Debating with More Persuasive LLMs Leads to More Truthful Answers
- A Comment On "The Illusion of Thinking": Reframing the Reasoning Cliff as an Agentic Gap