Does making an AI bigger actually teach it to guess what other people are thinking, or just fake it better?
Does scaling language models actually improve theory of mind reasoning abilities?
This explores whether bigger or more heavily trained language models actually get better at reasoning about what other people believe, want, and know (theory of mind), or whether they just get better at passing the tests for it.
This explores whether making language models bigger or training them harder makes them better at theory of mind, meaning the ability to track what someone else believes, wants, or knows, especially when that differs from reality. According to the corpus, scale helps in some ways and hurts in others. The more important finding is that accuracy scores are a poor guide to which is happening.
The clearest evidence that scale matters comes from reinforcement learning experiments on social reasoning Does reinforcement learning on theory of mind collapse with model scale?. At around 7B parameters, models trained this way developed explicit, step-by-step tracking of who believes what, and that skill carried over to new tasks. Smaller models reached similar accuracy by taking shortcuts, and their reasoning traces didn't show any real belief tracking. You could only see the difference by reading the step-by-step outputs. So size seems to set a minimum capacity before real mental-state reasoning can appear, but scoreboards alone won't tell you whether a model has crossed it.
This matters because the benchmarks themselves may be leaky. Plain supervised fine-tuning matches reinforcement learning on standard theory-of-mind tests, which suggests models are exploiting templated patterns and dataset quirks rather than reasoning about minds Can language models solve ToM benchmarks without real reasoning?. When the tests are open-ended, like real persuasion threads or messy multi-party conversations, LLMs fall back on surface strategies. Hybrid systems that force explicit belief tracking beat LLMs on their own, which hints that the gap comes from architecture, not just training Do large language models genuinely simulate mental states?. A broader reasoning study fits this picture: models break down on unfamiliar instances rather than harder ones, because they learn patterns from similar examples rather than general procedures Do language models fail at reasoning due to complexity or novelty?. More scale buys more familiar territory, and that can look like more understanding.
The most surprising result runs against the scaling story. Newer reasoning-focused models like o1 and Claude 3.7 Sonnet did worse than older models on a game-based test of false belief and perspective shifts, and sometimes worse than a simple word-embedding baseline Why do reasoning models fail at theory of mind tasks?. Optimizing for formal, math-style reasoning may actively wear away social reasoning. Visible chain-of-thought may not help either if it mostly imitates the form of reasoning seen in training data Does chain-of-thought reasoning reveal genuine inference or pattern matching?.
Putting it together: scaling looks necessary but not sufficient, and the type of training matters as much as the size. The question underneath is what would count as "real" theory of mind in a model at all. One philosophical position argues it's reasonable to credit LLMs with modest belief-like states without claiming anything about consciousness Can we defend modest mental attributions to large language models?. If you take that view, the useful question shifts from whether models have theory of mind to which training conditions produce reasoning you can actually inspect, rather than a lucky pattern match.
Sources 7 notes
7B models develop explicit, transferable belief-tracking under RL, while smaller models achieve comparable accuracy through shortcut learning that lacks interpretable reasoning traces. The mismatch between accuracy and reasoning quality is invisible without inspecting step-by-step outputs.
Supervised fine-tuning matches reinforcement learning performance on ToM tasks, suggesting models exploit structural vulnerabilities rather than develop genuine reasoning. Distribution biases and templated artifacts allow surface-level pattern recognition to achieve competitive generalization.
ChangeMyView and FANTOM benchmarks show LLMs fail at authentic perspective-taking in open-ended scenarios, despite succeeding on structured tasks. Hybrid Bayesian architectures that force explicit belief tracking outperform LLM-alone approaches, suggesting the gap is architectural rather than merely training-based.
LRMs don't break at complexity thresholds but at instance-novelty boundaries. Models fit instance-based patterns rather than generalizable algorithms, so any reasoning chain succeeds if trained on similar instances, regardless of length.
Claude 3.7 Sonnet and o1 fail measurably at Decrypto benchmark tasks testing representational change, false belief, and counterfactual reasoning—tasks where they score worse than both humans and simple word-embedding baselines. The evidence suggests formal reasoning optimization actively degrades social reasoning capability.
Show all 7 sources
CoT works by constraining models to reproduce familiar reasoning patterns from training, not by enabling novel symbolic reasoning. Performance degrades predictably under distribution shifts—the signature of imitation rather than capability emergence.
Both robustness and etiological deflationist arguments beg the question against inflationism. A graded approach ascribing metaphysically undemanding states like beliefs and desires—while withholding consciousness claims—mirrors how we treat non-human animals.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Do Theory of Mind Benchmarks Need Explicit Human-like Reasoning in Language Models?
- Evaluating Theory of Mind in Reasoning Models: Robustness over Reasoning
- A Systematic Review on the Evaluation of Large Language Models in Theory of Mind Tasks
- Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey
- Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens
- Deflating Deflationism: A Critical Perspective on Debunking Arguments Against LLM Mentality
- Evaluating Large Language Models in Theory of Mind Tasks
- Ring-Zero: Scaling Zero RL to a Trillion Parameters for Emergent Reasoning