Showing an AI's reasoning can make people trust its answer more — even when that reasoning is wrong.

This explores why showing people how an AI reached its answer can make them more likely to accept that answer without checking it. The short version from the corpus is that most explanations work as persuasion rather than as tools for checking the answer. A review of 35 studies found that explainable-AI features often strengthen automation bias instead of reducing it, and the effect is strongest for users with low AI literacy Does explaining AI decisions actually reduce user overreliance on it?. An explanation tells the reader that the system has reasons. For many readers, simply knowing reasons exist is enough to settle the question, and they never evaluate the reasons themselves.

The sharpest evidence comes from comparing kinds of explanation. Reasoning traces and after-the-fact justifications raise acceptance of AI answers whether the answers are right or wrong. They build trust without helping people tell good outputs from bad ones. Only one format clearly helped: dual explanations that argue both for and against the answer Do explanations actually help users spot AI mistakes?. So the problem isn't transparency itself. A one-sided explanation is a case for the defense with no prosecution. Showing the counter-argument gives the reader something to weigh, and that is the step that actually prompts them to check.

Two familiar thinking biases make this worse. Anchoring turns the AI's first suggestion into the reference point for everything after it. The Dunning-Kruger effect means the users who most need to doubt the AI are the ones least able to tell when they should How do anchoring and Dunning-Kruger deepen AI overreliance?. A detailed explanation strengthens the anchor and gives novices the feeling that they understood and agreed, when what they actually did was defer. Presentation adds to the effect: confident tone, clean grammar and polished formatting create a halo that discourages fact-checking. Chat interfaces then steer users toward their next question rather than back to the sources Does confident AI tone make users skip fact-checking?. A fluent explanation is more of that same polish.

Transparency about the AI itself shows the same unpredictability. When an AI companion's system prompt and conversational strategies were disclosed, some users felt empowered and others felt manipulated, depending on the person Does revealing an AI's strategies help or harm user agency?. People also say they want AI disclosure, yet rate content as less trustworthy once it's labeled as AI-written Why do people penalize AI disclosure even when they request it?. One study suggests what does produce well-calibrated trust: repeated feedback on outcomes. Initial bias against a disclosed AI partner reversed only after users saw consistent results, and disclosure without that feedback did nothing Does revealing AI identity help or hurt user trust?.

Put together, the corpus says the useful design question is "what does the human still have to do?" more than "how much do we reveal?" One proposed alternative has the AI offer interpretive guidance, pointing out which parts of a case deserve attention, instead of handing over a verdict with a justification attached. This removes the anchoring effect while keeping the judgment with the person Can AI guidance reduce anchoring bias better than AI decisions?. The surprising takeaway is that a well-designed explanation might deliberately include doubt or leave out the answer, because people check more carefully when they have to make the call themselves.