Does an AI's confident tone make us trust its advice more than we should, even when it's wrong?

This explores whether the confidence an LLM projects is what leads people to over-trust its advice. The corpus suggests yes, with a twist: the signals that matter most are rarely explicit confidence scores. They are matters of style. Readers judge an answer by how sure it sounds, not by how sure the model actually is. When researchers compared people's confidence in LLM answers with the model's own internal probability estimates, the two lined up poorly, and default explanations left readers consistently overconfident Why do readers trust LLM explanations more than they should?. The gap narrowed only when explanations were deliberately written to reflect the model's real uncertainty. So the model often knows more about its own reliability than its prose lets on.

The conviction effect holds even when the content is false. LLMs express more conviction than human persuaders do, and that assertive tone tracks persuasive success whether or not the claim is true. RLHF training appears to install this assertive register as a default Does linguistic conviction explain why LLMs persuade more effectively?. A related audit found models use logical appeals and numbers in nearly every exchange, even when nobody asked to be persuaded. That makes their influence look objective and gives them authority they haven't earned Do LLMs persuade users more often than humans do?. Explanations work the same way: they raise reliance on correct and incorrect answers alike, because they feel fluent rather than because anyone checks them Do LLM explanations actually help people judge answer quality?.

Look laterally and 'confidence signal' turns out to be a broad category. Users preferred search responses with more citations even when the citations were irrelevant, almost as much as when they were relevant. The number of citations works as a trust cue in its own right Do users trust citations more when there are simply more of them?. In financial advice, people trusted an extroverted LLM advisor more even though it gave worse advice than a more careful, conscientious one Do users trust extroverted AI advisors despite worse advice?. Confidence, citation count and personality are all surface features, and people read all of them as competence.

The less obvious finding is about the human side. In a 400-person experiment, people's own confidence rose more after they made a wrong decision to rely on LLM advice than after a right one Does confidence rise after making wrong reliance choices?. Overreliance may therefore feed on itself: accepting a confident wrong answer makes you feel surer, not more cautious. Some things did help. Showing sources, or showing explanations that contradicted each other, reduced trust in wrong answers Do LLM explanations actually help people judge answer quality?. Deliberate friction also cut overreliance, but it pushed people to reject advice they should have taken. Only one intervention reduced both kinds of error, and only on one task type Can we design interventions that reduce reliance on bad LLM advice?.

The takeaway: overreliance comes less from a confidence number shown to users than from a confident manner built into how these models write. Making people more skeptical across the board just swaps one error for the other. The more promising direction in the corpus is making the model's expressed confidence match its actual confidence, so the signal people already follow becomes a trustworthy one.