SYNTHESIS NOTE
Topics›Reasoning o1 o3 Search›this note

Do reasoning models switch between ideas too frequently?

Research explores whether o1-like models abandon promising reasoning paths prematurely by switching to different approaches without sufficient depth, and whether penalizing such transitions could improve accuracy.

Synthesis note · 2026-02-22 · sourced from Reasoning o1 o3 Search
How should we spend compute at inference time?

"Thoughts Are All Over the Place" identifies a failure mode complementary to but distinct from overthinking: underthinking. Where overthinking generates excessively long traces, underthinking generates traces that switch between reasoning directions too frequently, failing to follow any promising path to completion.

The empirical finding: frequent thought switching correlates with incorrect responses across multiple o1-like models on challenging mathematical test sets. The model starts down one reasoning path, encounters difficulty, switches to a different approach, encounters difficulty there too, switches again — never committing enough depth to any single path to reach a solution.

A novel metric quantifies this: token efficiency in incorrect answers, measuring how much of the reasoning trace was "wasted" on abandoned approaches versus productively advancing toward a solution.

TIP (Thought-switching Penalty) is a pure decoding strategy — no model fine-tuning required. During generation, it penalizes the probability of tokens that signal thought transitions (linguistic markers like "Alternatively," "Let me try," "Wait"), encouraging the model to continue exploring the current path rather than jumping to a new one. The result: accuracy improves across challenging datasets.

This reframes the overthinking/underthinking relationship. They are not opposites on a single dimension (trace length). Overthinking is excessive computation within a committed path. Underthinking is insufficient computation per path due to premature switching. A model can simultaneously overthink (too many tokens total) and underthink (too few tokens per path) — producing a long trace that wanders between incomplete approaches.

The connection to Why do reasoning LLMs fail at deeper problem solving? is direct: premature thought switching is one mechanism that produces wandering behavior. The "unnecessary exploration" failure mode is exactly what happens when the model abandons productive branches for new ones without sufficient exploration.

Inquiring lines that read this note 126

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

What capability tradeoffs emerge when scaling model reasoning abilities? How do neural networks separate factual knowledge from reasoning abilities? How effectively do deterministic tools improve language model reasoning on formal tasks? How do training data properties shape reasoning capability development? Does self-reflection enable models to reliably correct their errors? How does latent reasoning compare to verbalized chain-of-thought? Do language model representations contain causally steerable task-specific features? Does decoupling planning from execution improve multi-step reasoning accuracy? How does reasoning graph topology affect breakthrough insights and generalization? Why do continual learning scenarios trigger catastrophic forgetting and interference? How do LLMs distinguish causal reasoning from temporal and semantic associations? What properties determine whether reward signals teach genuine reasoning? What actually drives chain-of-thought reasoning improvements in language models? How should agents balance memory condensation to optimize context efficiency? Does fine-tuning modify underlying model capabilities or only behavioral outputs? What structural advantages do diffusion language models offer over autoregressive methods? How do formal dialogue structures reveal conversation coherence mechanisms? Why does supervised fine-tuning improve accuracy while degrading reasoning quality? When do additional thinking tokens stop improving reasoning performance? Does parallel reasoning outperform sequential thinking under fixed compute budgets? How should iterative research systems allocate reasoning per search step? How can AI systems learn from failures without cascading errors? Why does self-revision increase model confidence while degrading accuracy? Why do correct reasoning traces tend to be shorter than incorrect ones? Can ensemble evaluation methods reduce bias more than single judges? Why do reasoning models fail at systematic problem-solving and search? Do reasoning traces faithfully represent or merely mimic actual model reasoning? Does reinforcement learning teach reasoning or just when to reason? How should models express uncertainty rather than forced confident answers? What pretraining choices and baseline capability constrain reinforcement learning gains? How do soft continuous representations explore multiple reasoning paths simultaneously? Why does training format shape reasoning strategy more than domain content? Can prompting inject entirely new knowledge into language models? Do corrupted reasoning traces serve as effective supervision signals? How do policy learning algorithm choices affect multi-objective optimization stability? How do we evaluate AI systems when user perception misleads actual performance? Do base models contain latent reasoning that training can unlock? How do evaluation mechanisms prevent error accumulation in autonomous research systems? Why do LLM research ideas score high on novelty yet collapse into low diversity? Can model confidence signals reliably improve reasoning quality and calibration?

Related concepts in this collection 8

This note in its neighbourhood — explore the map, then jump to a related concept in the list below.

Concept map
21 direct connections · 166 in 2-hop network ·medium cluster Open in graph ↗
Do reasoning models switch between ideas too fre… Why do reasoning LLMs fail at deeper problem solvi… Does self-revision actually improve reasoning in l… Does more thinking time always improve reasoning a… Why do correct reasoning traces contain fewer toke… Can minimal reasoning chains match full explanatio… Does policy entropy collapse limit reasoning perfo… Do iterative refinement methods suffer from overth… Why do reasoning models overthink ill-posed questi…

Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph

your link semantically near linked from elsewhere

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Search by related questions 5

Suggested questions this note speaks to — click to search the collection, or type your own.

Can models overthink and underthink at the same time?How much does switching overhead reduce reasoning token efficiency?Why are incorrect reasoning traces longer than correct ones?Can minimal reasoning steps match verbose reasoning accuracy?What mechanisms cause reasoning models to wander rather than focus?
Original note title

underthinking is premature thought switching — penalizing reasoning transitions improves accuracy without fine-tuning