What makes an AI's explanation actually land — is it how detailed it is, or who's reading and what they need to decide?
What makes explanations appropriate for different task complexity and user expertise?
This explores what makes an AI explanation fit its situation, meaning how hard the task is and how much the person reading already knows, and whether the corpus has research on matching explanations to those two factors.
This explores what makes an AI explanation fit its situation: how hard the task is, and how much the person reading it already knows. The corpus has no study that crosses task complexity with user expertise directly. What it does have changes the question. It suggests that whether an explanation is appropriate depends less on how detailed it is and more on the situation it lands in and what it pushes the reader to do.
The main reframe comes from the argument that explainable AI is a communication problem, not a transparency problem What if XAI is fundamentally a communication problem?. The same explanation can work or fail depending on who presents it, how it is framed, and what role the recipient plays, whether they are deciding, auditing or just curious. Seen this way, expertise is one feature of the recipient's role, not a dial you turn up or down. An expert auditor and an expert end-user may need very different explanations of the same output.
The most surprising finding is about what explanations actually do to people. Reasoning traces and after-the-fact justifications tend to make users accept AI answers more, whether the answers are right or wrong Do explanations actually help users spot AI mistakes?. Only 'dual' explanations helped people tell correct outputs from incorrect ones. These lay out the case for an answer and the case against it. So a fluent, confident walkthrough can be the wrong choice exactly where the stakes are highest. The right explanation for a hard, high-risk task may be one that argues with itself.
Research on model reasoning offers a useful parallel, with a caveat: these studies are about how models think, not how people read. Chain-of-thought accuracy follows an inverted U. The best length grows as the task gets harder and shrinks as the model gets more capable Why does chain of thought accuracy eventually decline with length?. If that pattern carries over to people, a hypothesis the corpus doesn't test, harder tasks would call for longer explanations and more expert readers for shorter ones. Chain of Draft also found that about 92% of the tokens in a verbose reasoning chain served style and documentation rather than computation Can minimal reasoning chains match full explanations?. That is a reminder that much of a long explanation is written for a reader, so who that reader is matters. A related warning: longer reasoning doesn't reliably signal a harder problem. Trace length mostly reflects how familiar the problem looks Does longer reasoning actually mean harder problems?, and failures track novelty more than complexity Do language models fail at reasoning due to complexity or novelty?. So 'task complexity' may be the wrong measure for sizing an explanation. 'How unfamiliar is this case?' may be the better one.
One more parallel: systems that break a complex task into steps and show each step only the context it needs work better Can algorithms control LLM reasoning better than LLMs alone?. That points to tailoring by leaving things out, not just by simplifying. The corpus doesn't yet offer direct evidence on how novices and experts respond differently to the same explanation. Treat the length and decomposition points as hypotheses worth testing, not settled findings.
Sources 7 notes
Explanation quality is not intrinsic to the explanation itself but depends on the rhetorical situation: who presents it, how it is framed, and what role the recipient plays. Evaluations that ignore this triad measure only a narrow slice of real-world effectiveness.
Reasoning traces and post-hoc explanations increase user acceptance of AI answers regardless of correctness, engendering false trust. Only dual explanations presenting arguments for and against the answer genuinely help users distinguish correct from incorrect outputs.
Task accuracy peaks at intermediate CoT length, with optimal length increasing alongside task difficulty but decreasing with model capability. RL training naturally gravitates toward shorter chains as models improve, revealing that simplicity emerges from reward signals rather than explicit training.
Chain of Draft achieves equivalent accuracy to standard chain-of-thought on arithmetic, symbolic, and commonsense tasks while using only 7.6% of tokens. The 92.4% of removed tokens served style and documentation, not computation.
Controlled A* maze experiments show trace length correlates with difficulty only in-distribution but decouples entirely out-of-distribution. Trace length primarily reflects recall of training schemas, not adaptive computation.
Show all 7 sources
LRMs don't break at complexity thresholds but at instance-novelty boundaries. Models fit instance-based patterns rather than generalizable algorithms, so any reasoning chain succeeds if trained on similar instances, regardless of length.
LLM Programs embed LLMs within explicit algorithms that manage control flow and state, presenting only step-specific context to each LLM call. This information hiding addresses capability and context window limits while treating complex reasoning as modular, debuggable sub-tasks.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- When More is Less: Understanding Chain-of-Thought Length in LLMs
- Explaining Too Much? Understanding How Large Language Model Reasoning Traces Influence Performance and Metacognition
- Break the Chain: Large Language Models Can be Shortcut Reasoners
- The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity
- Think Deep, Not Just Long: Measuring LLM Reasoning Effort via Deep-Thinking Tokens
- Exploring automation bias in human-AI collaboration: a review and implications for explainable AI
- Fostering Appropriate Reliance on Large Language Models: The Role of Explanations, Sources, and Inconsistencies
- Large Language Model Reasoning Failures