AI models can think themselves into wrong answers — past a certain point, more thinking time actually lowers accuracy.
What happens when models overthink during test-time search?
This explores the failure modes that appear when a model spends more inference-time compute — more thinking tokens, more search steps, more revision passes — and why more effort past a certain point starts to hurt instead of help.
This explores what actually breaks when a model keeps thinking, searching, or revising past the point of usefulness — and the corpus is unusually direct about it: more compute is not free, and beyond a task-specific threshold it actively degrades the answer. The clearest evidence is the non-monotonic scaling curve: accuracy climbs, peaks, then falls sharply — in one study from 87.3% down to 70.3% as thinking tokens went from ~1,100 to 16,000 When does thinking too much actually hurt reasoning?. The mechanism isn't that the model runs out of ideas; it's that extended thinking inflates output variance and injects self-revision errors, so the model talks itself out of correct answers.
What's striking is that this same failure architecture reappears at every level of granularity. Iterative refinement methods — where a model redrafts a whole response repeatedly — reproduce overthinking one level up: each pass accumulates noise without any guarantee of improvement, the exact same trap as token-level rambling Do iterative refinement methods suffer from overthinking?. And external search frameworks aren't immune either: whether you use best-of-N sampling or MCTS, snowball errors accumulate per step regardless of the algorithm, so the framework you pick matters far less than your total compute budget and how reliable your reward signal is Does the choice of reasoning framework actually matter for test-time performance?. Overthinking, in other words, is structural, not incidental — it shows up wherever you add iterations without a good stopping rule.
A big part of the problem is that models are never taught when to stop. Reasoning training optimizes for producing reasoning steps, so when a question is ill-posed or missing a premise, a reasoning model dutifully generates redundant paragraphs instead of noticing there's nothing to answer — while plainer non-reasoning models correctly flag it as unanswerable Why do reasoning models overthink ill-posed questions?. Overthinking has a mirror-image sibling, too: underthinking, where o1-style models abandon promising reasoning paths mid-exploration and thrash between ideas, wasting tokens on half-finished approaches Do reasoning models switch between ideas too frequently?. So the pathology cuts both ways — commit too early and you flail, commit too late and you second-guess a right answer into a wrong one.
The more interesting thread is that the corpus doesn't just diagnose overthinking, it treats the right amount of thinking as something you can steer at inference time. ReBalance reads a model's own confidence — variance and overconfidence — as a live signal to dial reasoning up during underthinking and down during redundant overthinking, without any retraining Can confidence patterns reveal overthinking versus underthinking?. A simpler decoding-time penalty on thought-switching tokens curbs the thrashing side Do reasoning models switch between ideas too frequently?. Both point at the same insight: the model often already contains the signal for when it's spinning its wheels; the trick is listening to it rather than burning a fixed budget.
Here's the counterintuitive payoff the corpus leaves you with: since search steps obey the same diminishing-returns scaling curve as thinking tokens Do search steps follow the same scaling rules as reasoning tokens?, the win isn't quieter models that think less — it's models that think *differently*. When inference is going to feed a search process, training a model to converge on one confident answer is exactly wrong; you want it to emit many varied-but-competent solutions so the search can explore and recombine modes that a single-track, entropy-collapsed policy can never reach at all Should training maximize diversity when models feed into search?. And rather than hiding a model's messy backtracking, journey learning trains on the whole failed-and-recovered trajectory, teaching more robust reasoning than clean shortcut solutions do Can models learn better by training on messy exploration paths?. Overthinking is what search looks like when it has no diversity to explore and no signal telling it when to quit.
Sources 9 notes
Empirical studies demonstrate non-monotonic scaling in test-time reasoning: accuracy peaks at a critical thinking-token count, then declines sharply (87.3% to 70.3% as tokens scale from 1,100 to 16,000). Extended thinking inflates output variance and introduces self-revision errors rather than improving solution quality.
Sequential revision methods share the same failure architecture as token-level overthinking: they accumulate noise without guaranteed improvement. Progressive Draft Refinement avoids this by compressing memory between iterations, outperforming longer reasoning traces at matched compute.
Information-theoretic analysis shows BoN and MCTS converge in reasoning accuracy when controlling for total compute. Snowball errors accumulate per step regardless of framework; mitigation depends on search scope and reward function reliability, not the specific algorithm.
Reasoning models generate redundant, lengthy responses to questions with missing premises while non-reasoning models correctly identify them as unanswerable. Training optimizes for producing reasoning steps but never teaches models when to disengage.
o1-like models frequently abandon reasoning paths mid-exploration, wasting tokens on incomplete approaches. A decoding-only penalty on thought-transition tokens (TIP strategy) discourages switching, improving accuracy on challenging math without model fine-tuning.
Show all 9 sources
ReBalance uses confidence variance and overconfidence as diagnostic signals to apply training-free steering vectors that reduce overthinking redundancy while promoting exploration during underthinking, improving accuracy across models from 0.5B to 32B parameters.
Deep research agents improve with more search steps in a pattern mirroring the reasoning-token relationship, with both exhibiting diminishing returns. This reveals a new inference-compute axis beyond model capability alone.
Vector Policy Optimization trains models to emit varied competent solutions rather than converging to one answer. This unlocks search procedures like evolutionary algorithms to explore and combine modes, solving problems that entropy-collapsed policies cannot reach at all.
Research shows that training on messy trajectories—failed attempts, self-correction, and backtracking—teaches more robust reasoning than training only on shortcut solutions. This approach models o1-style deep reasoning as search internalization rather than solution memorization.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Does Thinking More always Help? Understanding Test-Time Scaling in Reasoning Models
- The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity
- When More Thinking Hurts: Overthinking in LLM Test-Time Compute Scaling
- Think Deep, Not Just Long: Measuring LLM Reasoning Effort via Deep-Thinking Tokens
- Missing Premise exacerbates Overthinking: Are Reasoning Models losing Critical Thinking Skill?
- Large Language Models Think Too Fast To Explore Effectively
- Meta-Reasoner: Dynamic Guidance for Optimized Inference-time Reasoning in Large Language Models
- Rethinking External Slow-Thinking: From Snowball Errors to Probability of Correct Reasoning