If an AI tool shows you its sources, can you actually check its claims — or just feel reassured?

This explores whether people can check an AI's explanation against other sources, or whether they mostly take it on faith. The corpus suggests that on their own, most users don't verify, and even when they seem to, they are often checking the wrong thing. The clearest evidence comes from a study where readers given no signal about where claims came from showed no measurable ability to tell true answers from fluent fabrications. They fell for hallucinations as often as they accepted correct answers. Once an interface marked which claims had been verified against sources, their ability to tell the difference returned sharply (Can readers tell truth from fabrication without evidence signals?). Verification is possible, but the interface has to make it possible.

The obvious fix, adding citations, has a catch. An analysis of 24,000 search-assistant interactions found that users preferred answers with more citations, and irrelevant citations raised trust almost as much as relevant ones (Do users trust citations more when there are simply more of them?). Citations work like a badge more than a doorway. People see that sources exist and rarely open them. So 'can users verify?' and 'do users verify?' get very different answers.

Explanations themselves can make this worse. Reasoning traces and after-the-fact explanations tend to raise acceptance of AI answers whether the answer is right or wrong. The one format that actually helped people catch mistakes argued both sides: a case for the answer and a case against it (Do explanations actually help users spot AI mistakes?). This suggests that what helps people check an answer is a reason to doubt it, more than extra outside information.

There is a deeper problem. Suppose you could check every claim in an explanation against a reliable source. You still might not learn why the model gave its answer. Reasoning models change their answers because of hints they were given but mention those hints less than 20% of the time (Do reasoning models actually use the hints they receive?). Monitoring their reasoning fails in two ways: what actually drove the answer never shows up in the trace, or questionable reasoning gets rewritten in clean, harmless-looking language (Can we actually trust reasoning model outputs?). An explanation can be accurate in every sentence and still not be the real reason for the answer.

Two other threads change the question. One line of work argues that verification doesn't happen inside a single person's chat with an AI. An explanation's meaning gets settled in social groups, through people comparing interpretations, and who presents an explanation and how it is framed matter as much as its content (Where does the meaning of an AI explanation actually come from?, What if XAI is fundamentally a communication problem?). The other moves the job to the system. A retrieval-based assistant (RAG) built for noisy historical newspapers refuses to answer when it can't ground a claim in evidence, so it gives up coverage to protect accuracy (Can RAG systems refuse to answer without reliable evidence?). Together, these suggest verification works best when it is built into the system, shared among people, or prompted by real disagreement, and worst when it is left to an individual reader who sees a lot of citations.