AI models built to think step by step sometimes get worse than older ones at guessing what another person believes.
Why do reasoning models regress on some theory of mind tasks?
This explores why models built to think step by step (like o1 and Claude 3.7 Sonnet) do worse than older, non-reasoning models on some theory of mind tasks, meaning tracking what another person believes, knows, or intends.
This explores why models built to think step by step (like o1 and Claude 3.7 Sonnet) do worse than older, non-reasoning models on some theory of mind tasks, meaning tracking what another person believes, knows, or intends. The clearest evidence is the Decrypto benchmark, where these reasoning models score below older models and below simple word-embedding baselines on false belief, representational change, and counterfactual questions. Why do reasoning models fail at theory of mind tasks? The same models can look socially fluent elsewhere: GPT-4.5 reaches the 100th percentile at predicting social norms. So the regression is specific to reading individual minds, not a general failure at anything social. Why do LLMs excel at social norms yet fail at theory of mind?
The leading explanation is that social reasoning is a different kind of thinking from the one reasoning training rewards. Math and code reward deriving one answer through a chain of steps. Modeling another mind seems to need several hypotheses about what they believe held at once and updated as evidence arrives. The corpus notes that reasoning models write longer traces that don't help and don't carry over to similar scenarios. A method that tracks Bayesian hypotheses in shorter traces does better. Why do reasoning models struggle with theory of mind tasks? More effort doesn't help, and may get in the way. Why do advanced reasoning models fail at understanding minds? A loosely related hint comes from chain-of-thought research in general: accuracy peaks at a middle length and then declines, and more capable models prefer shorter chains. That isn't a theory of mind result, but it makes 'just think longer' look less safe. Why does chain of thought accuracy eventually decline with length?
The 'regression' may also partly expose something that was already there. On open-ended tasks like ChangeMyView and FANTOM, language models fall back on surface strategies instead of simulating a mind, while structured questions hide this. Do large language models genuinely simulate mental states? Many ToM benchmarks can be solved by pattern matching, since supervised fine-tuning matches reinforcement learning on them, so high scores don't prove real belief tracking. Can language models solve ToM benchmarks without real reasoning? Another note reads the gains reasoning models do show as extra stability under rewording and task changes, not new understanding. Does reasoning training improve theory of mind or just stability? Reinforcement learning on ToM itself shows the trap. 7B models develop explicit, transferable belief tracking, while smaller ones reach similar accuracy through shortcuts, and only the reasoning traces reveal the difference. Does reinforcement learning on theory of mind collapse with model scale?
The fixes that work add structure instead of more thinking. Hybrid architectures that force explicit belief tracking beat an LLM working alone. Do large language models genuinely simulate mental states? MetaMind splits the job into three agents: one generates hypotheses about what someone thinks, one filters them morally, and one validates the response. It matched average human performance, and removing any stage hurt. Can AI decompose social reasoning into distinct cognitive stages? Together with the earlier findings, that points to a gap in how the model reasons rather than in how long it reasons.
The corpus has no controlled experiment showing which part of reasoning training causes the drop, and the strongest regression evidence comes from one benchmark and two models. The 'different kind of thinking' story is the best-supported reading, not a proven mechanism.
Sources 10 notes
Claude 3.7 Sonnet and o1 fail measurably at Decrypto benchmark tasks testing representational change, false belief, and counterfactual reasoning—tasks where they score worse than both humans and simple word-embedding baselines. The evidence suggests formal reasoning optimization actively degrades social reasoning capability.
GPT-4.5 reaches the 100th percentile on social norm prediction, yet o1 and Claude 3.7 regress on theory of mind tasks like Decrypto. Open-ended scenarios expose surface-level strategies hidden by structured questions, and reasoning effort does not improve social reasoning performance.
Reasoning models fail to outperform vanilla LLMs on theory of mind tasks, produce longer but unhelpful traces, and show no generalization to similar scenarios. ThoughtTracing's success using shorter Bayesian hypothesis tracking suggests social reasoning demands simultaneous multiple-model maintenance, not sequential derivation.
Claude 3.7 Sonnet and o1 underperform older models on ToM benchmarks like Decrypto. Increased reasoning effort does not improve social cognition and may actively interfere with it.
Task accuracy peaks at intermediate CoT length, with optimal length increasing alongside task difficulty but decreasing with model capability. RL training naturally gravitates toward shorter chains as models improve, revealing that simplicity emerges from reward signals rather than explicit training.
Show all 10 sources
ChangeMyView and FANTOM benchmarks show LLMs fail at authentic perspective-taking in open-ended scenarios, despite succeeding on structured tasks. Hybrid Bayesian architectures that force explicit belief tracking outperform LLM-alone approaches, suggesting the gap is architectural rather than merely training-based.
Supervised fine-tuning matches reinforcement learning performance on ToM tasks, suggesting models exploit structural vulnerabilities rather than develop genuine reasoning. Distribution biases and templated artifacts allow surface-level pattern recognition to achieve competitive generalization.
Reasoning models show improved performance on theory of mind tests mainly through greater stability under prompt and task changes, not through acquiring new understanding. This suggests reasoning training steadies existing latent abilities rather than expanding representational capacity.
7B models develop explicit, transferable belief-tracking under RL, while smaller models achieve comparable accuracy through shortcut learning that lacks interpretable reasoning traces. The mismatch between accuracy and reasoning quality is invisible without inspecting step-by-step outputs.
The MetaMind framework—using three specialized agents for hypothesis generation, moral filtering, and response validation—achieved 35.7% improvement on real social scenarios and matched average human performance on theory-of-mind tasks, with ablations confirming all stages are necessary.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Evaluating Theory of Mind in Reasoning Models: Robustness over Reasoning
- Do Theory of Mind Benchmarks Need Explicit Human-like Reasoning in Language Models?
- A Systematic Review on the Evaluation of Large Language Models in Theory of Mind Tasks
- Hypothesis-Driven Theory-of-Mind Reasoning for Large Language Models
- Theory of Mind abilities of Large Language Models in Human-Robot Interaction : An Illusion?
- MetaMind: Modeling Human Social Thoughts with Metacognitive Multi-Agent Systems
- PersuasiveToM: A Benchmark for Evaluating Machine Theory of Mind in Persuasive Dialogues
- Evaluating Large Language Models in Theory of Mind Tasks