Giving an AI more time to think only pays off if it was trained to think that way first.
Does inference-time compute scaling require explicit reasoning traces or verifiable rewards?
This asks whether spending more compute at inference time only pays off when a model is explicitly thinking out loud (visible reasoning traces) or being graded by a checkable reward — or whether the corpus shows other paths.
This explores whether inference-time compute scaling depends on two specific ingredients — explicit reasoning traces and verifiable rewards — or whether the corpus shows the picture is broader. The short version: neither is strictly required, but what you got out of training matters more than what you spend at inference. Models that were trained with a reasoning protocol turn extra tokens into real gains, while models without it largely don't — more compute can't manufacture a capability the training never installed Can non-reasoning models catch up with more compute?. So inference scaling isn't a free dial; it amplifies a structure that has to already be there.
The most surprising finding is how many *different* axes all obey the same scaling curve, none of them requiring a literal step-by-step trace. Search budget in agentic research systems scales just like reasoning tokens — more search steps buy better answers along the same diminishing-returns curve, which reframes retrieval itself as a compute axis you can trade against thinking Does search budget scale like reasoning tokens for answer quality? Do search steps follow the same scaling rules as reasoning tokens? How does test-time scaling work for individual research agents?. You can also scale *width* instead of depth: sampling parallel latent trajectories explores the solution space without the serial latency of longer chains Can reasoning systems scale faster by exploring parallel paths instead?. And at the broadest level, inference compute can substitute for raw model size — a small model given more thinking time matches a much bigger one on hard prompts, which means pretraining and inference are interchangeable resources rather than separate ones Can inference compute replace scaling up model size?.
Where verifiable rewards come in is subtler than "required." Reward evaluation itself can be scaled at test time — letting a reward model reason before it scores raises its ceiling beyond simple outcome-based grading Can reward models benefit from reasoning before scoring?. But you don't need an external verifier to spend compute well. Step-level confidence filtering lets a model judge its own traces mid-flight, catching breakdowns and stopping early — matching majority-vote accuracy with far fewer generated traces, using the model's internal signal rather than a verifiable reward Does step-level confidence outperform global averaging for trace filtering?. The real lever turns out to be *allocation*: spending the same total budget adaptively — little on easy prompts, lots on hard ones — beats uniform spending and even beats bigger models under flat budgets Can we allocate inference compute based on prompt difficulty? How should we spend compute at inference time?.
Two notes keep this honest. Fluent reasoning isn't the same as solving: frontier reasoning models hit only ~20% on constraint-satisfaction problems that demand genuine backtracking, so visible traces can be long and confident yet competence-free Can reasoning models actually sustain long-chain reflection?. And the trace doesn't even have to accumulate — memoryless, Markov-style reasoning that contracts a problem step by step and forgets its history reaches the same answers without dragging the full chain along Can reasoning systems forget history without losing coherence?. So the answer is: inference-time scaling needs neither a literal reasoning transcript nor an external verifier. It needs a training-installed protocol that makes extra compute productive, and a smart policy for where to spend it — across thinking, search, width, or self-evaluation.
Sources 12 notes
Reasoning models persistently outperform non-reasoning models regardless of inference budget because training instills a reasoning protocol that makes additional tokens productive. The gap is fundamentally about deployment mechanisms and training structure, not raw capability.
Agentic deep research shows monotonic-to-diminishing-returns curves for search iterations, matching reasoning token scaling. This creates a new inference-compute axis: models can trade off reasoning budget against search budget to optimize answer quality.
Deep research agents improve with more search steps in a pattern mirroring the reasoning-token relationship, with both exhibiting diminishing returns. This reveals a new inference-compute axis beyond model capability alone.
Research shows that deep research agents exhibit test-time scaling laws where search steps scale similarly to reasoning tokens, and live search outperforms memorized retrieval on knowledge-intensive tasks. Data efficiency is extreme—78 curated demonstrations outperform 10K samples for agency.
GRAM demonstrates that recursive reasoning models should maintain and explore multiple latent trajectories in parallel, not only deepen single paths. Width-scaling avoids the serial latency penalty of depth while sampling the solution distribution more effectively on ambiguous problems.
Show all 12 sources
Snell et al. (2024) showed that inference-time compute trades off against model parameter scaling, especially on difficult prompts. This reveals pretraining and inference compute are not independent resources.
Three independent teams (RRM, RM-R1, DeepSeek-GRM) discovered that adding chain-of-thought reasoning before reward scoring enables adaptive test-time compute scaling for evaluation. Reasoning-based approaches raise the capability ceiling of reward models beyond what outcome-based evaluation achieves.
Local step-level confidence catches reasoning breakdowns that global averaging masks and enables early stopping before traces complete. This approach achieves comparable accuracy gains to naive majority voting with far fewer generated traces, proving trace quality matters more than quantity.
Research shows inference effectiveness varies dramatically by prompt difficulty. Reallocating the same total compute adaptively—giving easy prompts less and hard ones more—substantially outperforms larger models under uniform budgets.
Research shows that uniform inference budgets waste compute; allocation should vary by prompt. Test-time compute can substitute for training-time scaling on hard problems, but cannot overcome fundamental limitations set by the training regime.
DeepSeek-R1 and o1-preview achieve only 20-23.6% exact match on 850 constraint satisfaction problems requiring genuine backtracking. This ceiling reveals that reflective reasoning fluency does not translate to actual problem-solving competence on unfamiliar instance structures.
Atom of Thoughts decomposes problems into DAGs and contracts them iteratively, ensuring each state depends only on the current problem—not prior steps. This memoryless approach eliminates historical baggage that bloats reasoning while maintaining answer equivalence.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- When More Thinking Hurts: Overthinking in LLM Test-Time Compute Scaling
- Reasoning Models Can Be Effective Without Thinking
- ZebraLogic: On the Scaling Limits of LLMs for Logical Reasoning
- Learning to Think: Information-Theoretic Reinforcement Fine-Tuning for LLMs
- Does Thinking More always Help? Understanding Test-Time Scaling in Reasoning Models
- The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity
- Think Twice: Enhancing LLM Reasoning by Scaling Multi-round Test-time Thinking
- From Web Search towards Agentic Deep Research: Incentivizing Search with Reasoning Agents