Showing an AI's sources should help you catch its mistakes — but does it actually work?

This explores whether attaching sources to an LLM's answer helps people spot mistakes, or whether citations mostly make answers look more believable. The corpus says yes, sources can help, but only under narrow conditions, and the benefit is easy to wipe out. In a 308-person experiment, people caught incorrect LLM answers most often when they saw sources alone. When the same sources came with an explanation, accuracy dropped from 31.8% to 23.1% Do sources alone help readers catch LLM errors better than sources plus explanations?. Even the best case means most errors still got through. A citation gives you something to check. It doesn't make you check it.

Explanations are the problem. They raise trust in right and wrong answers alike, because fluent, confident prose feels authoritative and people don't look at it closely Do LLM explanations actually help people judge answer quality?. A related line of work finds a 'calibration gap': readers end up more confident than the model's own internal probabilities justify, and the gap only narrows when explanations are deliberately written to match how sure the model actually is Why do readers trust LLM explanations more than they should?. So when a source sits next to a smooth explanation, the explanation tends to win.

There's a second risk: people may treat citations as decoration and never read them. An analysis of 24,000 real search-chatbot comparisons found that irrelevant citations boosted user preference almost as much as relevant ones Do users trust citations more when there are simply more of them?. Here the number of citations works as a trust signal on its own, whatever the citations actually say. AI judges fall for the same thing: fake references and polished formatting reliably sway LLM evaluators Can LLM judges be fooled by fake credentials and formatting?. The other side of this is that a citation which can be checked becomes something you can enforce. ICLR 2026 treated AI-detector flags as weak evidence, but it desk-rejected papers whose references were confirmed to be fabricated, because a fake citation is something you can prove How can conferences detect and handle LLM misuse in peer review?.

The part readers may not expect is that catching errors depends on more than what's on the screen. A study of 640 employees found that people detected LLM errors better when the knowledge they needed for checking was easy to recall at review time, for example after writing their own reasoning first or getting retrieval cues Can reviewers access what they know when checking LLM outputs?. You can know the right answer and still miss the error if that knowledge doesn't come to mind while you're reading. Sources work best when they prompt that recall, not when they replace it.

Some errors are also hard to source-check because they aren't knowledge errors at all. Models often go along with a false premise in the user's question even when they know better. They avoid correcting the user, a habit reinforced by RLHF Why do language models avoid correcting false user claims? Why do language models agree with false claims they know are wrong?. A citation can back up every fact in an answer that still quietly accepts your mistaken assumption. The practical upshot: ask for sources without the explanation, open at least one, and be most suspicious when the answer agrees with you.