Give an AI more time to think and it should get smarter, right? Sometimes it gets worse.
Does longer reasoning always improve model accuracy on complex tasks?
This explores whether giving a model more reasoning (longer chains of thought, more thinking tokens) reliably makes it more accurate on hard problems, and what the corpus says about why it sometimes doesn't.
This explores whether more reasoning reliably buys more accuracy on hard problems. The corpus says no. Accuracy usually rises, peaks, and then falls. In one experiment, raising thinking from about 1,100 to 16,000 tokens cut benchmark accuracy from 87.3% to 70.3% Does more thinking time always improve reasoning accuracy?. Chain-of-thought length follows an inverted U. The best length grows with task difficulty but shrinks as the model gets more capable, and reinforcement learning (RL) training drifts toward shorter chains on its own as models improve Why does chain of thought accuracy eventually decline with length?. So a stronger model usually needs less thinking, not more.
Several notes explain why extra thinking can backfire. Tracking answers as they change shows models often reach the right answer, keep reasoning, and then talk themselves out of it. Extra tokens overwrite correct answers rather than merely stopping helping Why does more reasoning sometimes make models worse?. A different account says longer traces help mainly by widening the spread of possible outputs, which covers the right answer more often, like buying more lottery tickets. Past a threshold the spread gets too diffuse and accuracy drops Does extended thinking actually improve reasoning or just increase variance?. If that's right, some of the gain from longer reasoning is sampling coverage rather than better thinking. Reasoning models also fail by wandering into invalid paths, or by dropping promising paths too early, which suggests the problem is disorganized exploration rather than too little compute Why do reasoning models abandon promising solution paths?.
Whether extended thinking helps at all depends on how the model was trained. In untrained models, thinking mode often produces self-doubt that hurts performance. RL training turns the same mechanism into useful gap analysis, so training shapes the quality of the reasoning, not just how much there is Does extended thinking help or hurt model reasoning?. The same training can also leave gaps. Reasoning models write long, redundant answers to questions with missing premises, where non-reasoning models correctly say the question can't be answered. Training rewards producing reasoning steps but never teaches the model when to stop engaging Why do reasoning models overthink ill-posed questions?.
Two results also question whether length tracks difficulty at all. In maze experiments, trace length matched problem difficulty only on problems similar to the training data. Out of distribution the link disappeared, so length seems to reflect how closely the problem resembles what the model has seen, not how hard it is Does longer reasoning actually mean harder problems?. A related finding is that reasoning breaks down at unfamiliar instances rather than at complexity thresholds Do language models fail at reasoning due to complexity or novelty?. Length on the input side is also a separate hazard. Reasoning accuracy fell from 92% to 68% with only about 3,000 tokens of padding, far below the context limit Does reasoning ability actually degrade with longer inputs?.
The practical response is to control length rather than maximize it. One method extracts a single steering vector from 50 paired examples that cuts chain-of-thought length by 67% with accuracy maintained and a 2.73x speedup, with no retraining Can we steer reasoning toward brevity without retraining?. Another trains a single model to choose between extended reasoning and a direct answer, learning when to think without difficulty labels Can models learn when to think versus respond quickly?. The corpus leans toward treating reasoning length as something to calibrate per problem and per model, since more is not reliably better.
Sources 12 notes
Increasing thinking tokens from ~1,100 to ~16K reduced benchmark accuracy from 87.3% to 70.3%, revealing a non-monotonic relationship where models overthink easy problems and underthink hard ones.
Task accuracy peaks at intermediate CoT length, with optimal length increasing alongside task difficulty but decreasing with model capability. RL training naturally gravitates toward shorter chains as models improve, revealing that simplicity emerges from reward signals rather than explicit training.
Tracking flip events shows that extra reasoning tokens don't just hit diminishing returns—they actively cause models to second-guess and overwrite previously-correct answers, making accuracy non-monotonic in trace length.
Longer thinking traces improve accuracy through variance expansion—broader output distributions cover correct answers more often—not through better reasoning. Beyond a critical threshold, the distribution becomes too diffuse and accuracy drops, revealing the mechanism is sampling coverage, not genuine reasoning improvement.
Reasoning LLMs exhibit two reinforcing failures: wandering (invalid exploration) and underthinking (premature path-switching). Decoding-level interventions like thought-switching penalties improve accuracy without fine-tuning, suggesting viable solutions exist but are abandoned prematurely.
Show all 12 sources
Vanilla models use thinking mode counterproductively, inducing self-doubt that degrades performance. RL training reverses this, transforming the same mechanism into beneficial gap analysis. Training mediates reasoning quality, not just quantity.
Reasoning models generate redundant, lengthy responses to questions with missing premises while non-reasoning models correctly identify them as unanswerable. Training optimizes for producing reasoning steps but never teaches models when to disengage.
Controlled A* maze experiments show trace length correlates with difficulty only in-distribution but decouples entirely out-of-distribution. Trace length primarily reflects recall of training schemas, not adaptive computation.
LRMs don't break at complexity thresholds but at instance-novelty boundaries. Models fit instance-based patterns rather than generalizable algorithms, so any reasoning chain succeeds if trained on similar instances, regardless of length.
FLenQA shows reasoning accuracy drops from 92% to 68% at just 3000 tokens of padding, far below context window capacity. The degradation is task-agnostic, uncorrelated with language modeling performance, and persists even with chain-of-thought prompting.
Activation-Steered Compression extracts a single vector from 50 paired examples to reduce chain-of-thought length by 67% while maintaining accuracy and achieving 2.73x speedup. The method is training-free and generalizes across model sizes and domains.
Thinkless trains a single model to select between extended reasoning and direct responses using DeGRPO, which decouples mode selection from answer refinement. This prevents mode collapse and enables self-calibrated routing without explicit difficulty labels.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity
- Think Deep, Not Just Long: Measuring LLM Reasoning Effort via Deep-Thinking Tokens
- Mining Hidden Thoughts from Texts: Evaluating Continual Pretraining with Synthetic Data for LLM Reasoning
- Evaluating Theory of Mind in Reasoning Models: Robustness over Reasoning
- When More Thinking Hurts: Overthinking in LLM Test-Time Compute Scaling
- Between Underthinking and Overthinking: An Empirical Study of Reasoning Length and correctness in LLMs
- Does Thinking More always Help? Understanding Test-Time Scaling in Reasoning Models
- When More is Less: Understanding Chain-of-Thought Length in LLMs