SYNTHESIS NOTE
Do reasoning models switch between ideas too frequently?
Research explores whether o1-like models abandon promising reasoning paths prematurely by switching to different approaches without sufficient depth, and whether penalizing such transitions could improve accuracy.
"Thoughts Are All Over the Place" identifies a failure mode complementary to but distinct from overthinking: underthinking. Where overthinking generates excessively long traces, underthinking generates traces that switch between reasoning directions too frequently, failing to follow any promising path to completion.
The empirical finding: frequent thought switching correlates with incorrect responses across multiple o1-like models on challenging mathematical test sets. The model starts down one reasoning path, encounters difficulty, switches to a different approach, encounters difficulty there too, switches again — never committing enough depth to any single path to reach a solution.
A novel metric quantifies this: token efficiency in incorrect answers, measuring how much of the reasoning trace was "wasted" on abandoned approaches versus productively advancing toward a solution.
TIP (Thought-switching Penalty) is a pure decoding strategy — no model fine-tuning required. During generation, it penalizes the probability of tokens that signal thought transitions (linguistic markers like "Alternatively," "Let me try," "Wait"), encouraging the model to continue exploring the current path rather than jumping to a new one. The result: accuracy improves across challenging datasets.
This reframes the overthinking/underthinking relationship. They are not opposites on a single dimension (trace length). Overthinking is excessive computation within a committed path. Underthinking is insufficient computation per path due to premature switching. A model can simultaneously overthink (too many tokens total) and underthink (too few tokens per path) — producing a long trace that wanders between incomplete approaches.
The connection to Why do reasoning LLMs fail at deeper problem solving? is direct: premature thought switching is one mechanism that produces wandering behavior. The "unnecessary exploration" failure mode is exactly what happens when the model abandons productive branches for new ones without sufficient exploration.
Inquiring lines that read this note 133
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
What causes foundation models to develop task heuristics instead of understanding?- Why do foundation models develop heuristics instead of world models?
- Can explicit constraint statements override the dominance of surface heuristics?
- How do foundation models develop task-specific heuristics instead of world models?
- How do models integrate conflicting signals in reasoning tasks?
- Can inflection points in reasoning detect when models genuinely change their minds?
- What changes when reasoning models adopt trajectory-response output formats?
- Why do models skip steps that would make reasoning clearer?
- When does knowledge activation fail across different model architectures?
- Why does inference-time thinking hurt proactive critical thinking in vanilla models?
- Does this reasoning steering method work consistently across all model sizes?
- Can auxiliary modules preserve reasoning without catastrophic forgetting?
- Can we steer model reasoning by manipulating single features?
- How do two-phase training dynamics explain reasoning emergence?
- What makes some reasoning strategies genuinely novel versus latent?
- Why do single examples trigger large reasoning improvements in models?
- Can contrastive learning teach models to switch between logical and emotional reasoning?
- Can penalizing reasoning transitions fix underthinking without fine-tuning models?
- Why does fine-tuning degrade reasoning quality even as accuracy improves?
- Does fine-tuning models for specific tasks destroy their ability to reason?
- Why does reasoning fine-tuning reduce a model's ability to abstain?
- Does penalizing thought transitions improve reasoning without model retraining?
- Does fine-tuning push models toward reasoning shortcuts that bypass the chain entirely?
- Can reflection in reasoning models be corrective rather than just confirmatory?
- Why does self-revision degrade reasoning accuracy in o1-like models?
- Why does reflection in reasoning models stay confirmatory instead of corrective?
- Why does revision often make reasoning accuracy worse in frontier models?
- Why does reflection in reasoning models tend to be confirmatory rather than corrective?
- Why does reflection in reasoning models confirm rather than correct initial directions?
- Does internal self-revision actually degrade reasoning accuracy in models?
- Why does reflection in reasoning models mostly confirm the first answer?
- Why does reflection in reasoning models rarely overturn initial answers?
- Can step-level deliberation flags guide other reasoning systems?
- What happens to chain-of-thought performance across distribution shifts?
- Why do longer reasoning chains signal hesitation rather than depth?
- Why do longer reasoning chains correlate with lower accuracy in o1-like models?
- Why do some reasoning steps receive negligible attention from later steps?
- How does interaction horizon differ from chain-of-thought depth?
- How does making implicit reasoning requirements explicit change model performance?
- Why does per-step deliberation lose global perspective compared to dynamic discovery?
- Can a single model implement fast thinking, slow thinking, and tool use?
- Why might chain-of-thought reasoning bypass action selection pathways?
- What makes answer equivalence sufficient to discard a reasoning path?
- Can we detect redundant reasoning steps during model inference instead of training?
- How much of chain-of-thought reasoning is actually redundant?
- What makes bilevel metacognition architectural rather than emergent in current systems?
- What role does exploration-exploitation balance play in abstraction formation?
- Why do aha moments emerge specifically during the planning phase?
- How does early commitment in reasoning differ from early exploitation in planning?
- Why do contrastive reasoning approaches outperform single-path belief evaluation?
- Why does the same recalled information lead to different reasoning conclusions?
- How does graph of thoughts enable divide-and-conquer reasoning patterns?
- What makes multi-paradigm chaining a distinct reasoning topology?
- Why does reasoning graph topology evolve differently across training phases?
- What distinguishes systematic search from wandering exploration in reasoning?
- Can static reasoning patterns work better than dynamic branch selection?
- How do progressive abstraction chains differ from branching reasoning topologies?
- How does o1-style reasoning relate to learned search processes versus memorized solutions?
- Does the reversal curse stem from the same one-way commitment architecture?
- How does flip-event regression differ from premature thought path abandonment?
- What triggers overthinking versus underthinking in reasoning models?
- How does reasoning instability prevent models from modeling individuals?
- Does reasoning structure match explicit versus implicit task demands?
- Why do reasoning models verbalize reasoning shortcuts less than necessary?
- Do base models and reasoning models fail in opposite directions on uncertainty?
- Can explicit optimal algorithms prevent reasoning model collapse at high complexity?
- Why do reasoning models wander instead of searching systematically?
- Which constraint types do reasoning models handle best?
- Do reasoning models switch approaches when encountering local difficulty?
- Can models overthink and underthink at the same time?
- What mechanisms cause reasoning models to wander rather than focus?
- How do single wrong steps corrupt entire reasoning chains?
- What failure modes emerge when scheme classification feeds downstream reasoning pipelines?
- What causes reasoning quality to degrade during long research tasks?
- What role do cyclic fixed points play in stable reasoning?
- Why does convergence stability sometimes mislead about reasoning correctness?
- Why do even deterministic models show abrupt shifts across time?
- Do tool-use capabilities actually help models recover from errors in reasoning?
- Can parallel thinking outperform sequential thinking under the same token budget?
- What makes diverse reasoning sources more valuable than deeper single paths?
- When are multiple independent attempts more valuable than depth?
- Does unrestricted reasoning per search step degrade iterative quality over time?
- What makes multi-turn critique trajectories more effective than single-turn reasoning chains?
- How should iterative research tasks limit context per reasoning turn?
- Why do long-horizon reasoning tasks need per-turn step limits rather than just compute budgets?
- What is the optimal balance between search rounds and reasoning depth per round?
- Why do per-turn thinking budgets matter alongside iterative retrieval depth?
- Does reflection destabilize reasoning in dynamic environments?
- Does thought consolidation address the confirmatory reflection problem in reasoning models?
- Why do reasoning chains degenerate into undirected exploration at scale?
- Why do larger reasoning models show cyclicity only in later layers?
- Can layer-wise prediction stabilization identify when genuine reasoning has stopped?
- Why do longer reasoning chains explore like tourists instead of scientists?
- What makes o1's chain-of-thought processing specifically effective for exploration tasks?
- Why does reflection in reasoning models often become theater rather than genuine thought?
- Why does intermediate step quality predict reasoning outcomes better than global features?
- Why do wrong numbers cost less accuracy than shuffled reasoning steps?
- Why do reasoning traces fail to accurately reflect model decision-making?
- How does RL refine reasoning paths without simply adding model capability?
- Why do models follow a two-phase pattern of procedural then strategic learning?
- What happens to reasoning accuracy when models use more thinking tokens?
- How does soft thinking compare to sampling multiple independent reasoning paths?
- How much does switching overhead reduce reasoning token efficiency?
- Why do different model training approaches produce different overthinking thresholds?
- What happens to model reasoning accuracy as thinking token requirements exceed critical thresholds?
- Which tokens actually change across different reasoning paths in rollouts?
- How does continuous soft thinking explore multiple paths without explicit training?
- How do soft thinking and token-level mixtures explore multiple paths simultaneously?
- How do continuous concept tokens explore multiple reasoning paths without explicit sampling?
- What causes reasoning accuracy to degrade beyond a critical thinking-token threshold?
- What distinguishes redundant cycles from productive reconsidering cycles?
- What happens to iterative search quality when reasoning depth is unconstrained?
- What makes deterministic recursive reasoning models underperform on multi-solution tasks?
- Why does reapplying the same computation stages improve model performance?
Related concepts in this collection 8
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Concept map
Do reasoning models switch between ideas too fre…
Why do reasoning LLMs fail at deeper problem solvi…
Does self-revision actually improve reasoning in l…
Does more thinking time always improve reasoning a…
Why do correct reasoning traces contain fewer toke…
Can minimal reasoning chains match full explanatio…
Does policy entropy collapse limit reasoning perfo…
Do iterative refinement methods suffer from overth…
Why do reasoning models overthink ill-posed questi…
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
your link
semantically near
linked from elsewhere
-
Why do reasoning LLMs fail at deeper problem solving?
Explores whether current reasoning models systematically search solution spaces or merely wander through them, and how this affects their ability to solve increasingly complex problems.
underthinking via switching is one mechanism producing the wandering pattern
-
Does self-revision actually improve reasoning in language models?
When o1-like models revise their own reasoning through tokens like 'Wait' or 'Alternatively', does this reflection catch and fix errors, or does it introduce new mistakes? This matters because self-revision is marketed as a key capability.
self-revision is a specific form of switching: the model revises (switches away from) an answer rather than deepening its current approach
-
Does more thinking time always improve reasoning accuracy?
Explores whether extending a model's thinking tokens linearly improves performance, or if there's a point beyond which additional reasoning becomes counterproductive.
the threshold may partly reflect switching overhead: tokens spent on transitions rather than productive reasoning
-
Why do correct reasoning traces contain fewer tokens?
In o1-like models, correct solutions are systematically shorter than incorrect ones for the same questions. This challenges assumptions that longer reasoning traces indicate better reasoning, and raises questions about what length actually signals.
incorrect traces are longer partly because switching generates wasted tokens
-
Can minimal reasoning chains match full explanations?
Does removing all explanatory text from chain-of-thought reasoning preserve accuracy? This tests whether verbose intermediate steps are necessary for solving problems or just artifacts of how language models are trained.
CoD addresses underthinking from the format side: minimal per-step drafts enforce depth within each step by eliminating the verbose intermediate context that enables thought-switching; where TIP penalizes switching tokens at decoding time, CoD prevents the runway for switching in the first place
-
Does policy entropy collapse limit reasoning performance in RL?
As reinforcement learning models become more confident in their policy choices, entropy drops and performance plateaus. Can we identify and counteract this bottleneck to sustain scaling?
training-time analog: entropy collapse reduces exploration diversity during training (narrowing the strategy repertoire), while underthinking reduces exploration depth during inference (abandoning strategies prematurely); both are exploration-exploitation failures at different timescales
-
Do iterative refinement methods suffer from overthinking?
Iterative refinement approaches like Self-Refine structurally resemble token-level overthinking in o1-like models. Does revision across multiple inference calls reproduce the same accuracy degradation seen within single inferences?
timescale generalization: underthinking operates within a single inference call (switching between reasoning threads); iterative refinement reproduces the same switching pattern across multiple inference calls — TIP-like penalties on transition tokens may apply at both timescales
-
Why do reasoning models overthink ill-posed questions?
Explores why models trained for extended reasoning produce drastically longer, less useful responses to unanswerable questions—and whether this represents a fixable training deficit or inherent limitation.
distinct but related failure: underthinking is premature switching between approaches (too shallow per path); overthinking on missing premises is inability to disengage (no valid path exists); both reveal the model lacks metacognitive control over its reasoning allocation
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMs
- Meta-Reasoner: Dynamic Guidance for Optimized Inference-time Reasoning in Large Language Models
- Evaluating Theory of Mind in Reasoning Models: Robustness over Reasoning
- Test-time Prompt Intervention
- Large Language Models Think Too Fast To Explore Effectively
- Fast, Slow, and Tool-augmented Thinking for LLMs: A Review
- The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity
- LLMs Can Easily Learn to Reason from Demonstrations Structure, not content, is what matters!
Original note title
underthinking is premature thought switching — penalizing reasoning transitions improves accuracy without fine-tuning