When More Thinking Hurts: Overthinking in LLM Test-Time Compute Scaling

Scaling test-time compute through extended chains of thought has become a dominant paradigm for improving large language model reasoning. However, existing research implicitly assumes that longer thinking always yields better results. This assumption remains largely unexamined. We systematically investigate how the marginal utility of additional reasoning tokens changes as compute budgets increase. We find that marginal returns diminish substantially at higher budgets and that models exhibit “overthinking”, where extended reasoning is associated with abandoning previously correct answers. Furthermore, we show that optimal thinking length varies across problem difficulty, suggesting that uniform compute allocation is suboptimal. Our cost-aware evaluation framework reveals that stopping at moderate budgets can reduce computation significantly while maintaining comparable accuracy.

Introduction. Scaling inference-time compute through lengthy chains of thought has achieved remarkable success on mathematical reasoning benchmarks (DeepSeek-AI et al., 2025; Muennighoff et al., 2025). Recent work has established that test-time compute scaling can be more effective than model scaling for many tasks (Snell et al., 2024; Wu et al., 2025a). The prevailing assumption in this line of research is straightforward: more thinking leads to better answers. Models are encouraged to reason longer, with performance curves consistently showing accuracy improvements as token budgets increase. Yet the assumption that thinking length and answer quality are monotonically related has never been systematically examined. We challenge this assumption by drawing an analogy from economics: the law of diminishing marginal returns. Just as additional units of input eventually yield smaller increments of output, additional tokens of reasoning may provide progressively less benefit. More critically, extended thinking might even be harmful.

Discussion / Conclusion. We analyze diminishing returns in test-time compute scaling, finding that (1) marginal utility decreases substantially at high budgets, and (2) models exhibit “overthinking,” abandoning correct answers after extended reasoning. We introduce flip event tracking and cost-aware evaluation metrics to capture accuracy-compute trade-offs. We encourage the community to report efficiency frontiers alongside accuracy curves.