Making a smaller model think longer can match a bigger one — but only on problems hard enough to need it.
How does test-time compute substitute for model parameter scaling?
This explores the finding that you can spend compute at inference time instead of building a bigger model — and where that trade actually holds versus where it breaks.
This explores the trade where a smaller model thinking harder at inference time can stand in for a larger model. The cleanest version of this comes from Snell et al., who showed that on hard prompts, a small model given more inference compute can match a larger one — meaning pretraining compute and inference compute aren't separate budgets but partly interchangeable resources you can shift between Can inference compute replace scaling up model size?. The catch is in the phrase 'hard prompts': the substitution isn't free or universal, which is why the rest of the corpus is really about *when* and *how* the trade works.
The first thing to know is that 'more inference compute' isn't one knob. Test-time scaling splits into internal methods (training a model to reason autonomously, building capability) and external methods (search and verification at inference, extracting performance from capability already there) — and these complement rather than compete How do internal and external test-time scaling compare? How should test-time scaling methods be categorized and designed?. That distinction matters for the substitution question, because it sets a ceiling: a non-reasoning model can't simply spend its way up to a reasoning model's level no matter how large the inference budget, since the reasoning model was *trained* to make extra tokens productive Can non-reasoning models catch up with more compute?. So inference compute substitutes for parameters only once the model knows how to use it — it amplifies a protocol it already has rather than installing a missing one.
There's also a quieter mechanism worth knowing: the substitution can be smuggled into training. 'Thinking-augmented' pretraining bakes reasoning traces into the training data, hitting 3x data efficiency, with harder tokens automatically getting longer traces — essentially test-time compute allocation moved upstream into the pretraining mix Can training data augmentation match test-time compute scaling benefits?. This blurs the line: the same compute can buy you capability before deployment or performance during it.
When you do spend at inference, *how* you spend dominates how much. Adaptive allocation — more compute on hard prompts, less on easy ones — beats uniform budgets How should we spend compute at inference time?, and the big axis is parallel (sampling many shots for coverage) versus sequential (longer chains for depth), a genuine trade-off keyed to task structure How should we balance parallel versus sequential compute at test time?. Newer work pushes width via parallel latent trajectories to dodge the latency of pure depth Can reasoning systems scale faster by exploring parallel paths instead?. Notably, the *framework* matters less than total compute and reward quality — Best-of-N and MCTS converge once you control for budget Does the choice of reasoning framework actually matter for test-time performance?.
The most disorienting finding — and the one that reframes the whole substitution — is that extended thinking may not work the way it looks. Longer traces appear to raise accuracy largely by inflating output variance (a broader distribution covers the right answer more often), not by reasoning better; past a threshold the distribution gets too diffuse and accuracy *drops* Does extended thinking actually improve reasoning or just increase variance?. If that's right, inference compute often substitutes for parameters by buying *sampling coverage* rather than genuine extra cognition — which explains both why it works on hard prompts and why it has a ceiling. The same compute axis even generalizes beyond reasoning: in agentic systems, search steps follow the same scaling curve as reasoning tokens How does test-time scaling work for individual research agents?.
Sources 11 notes
Snell et al. (2024) showed that inference-time compute trades off against model parameter scaling, especially on difficult prompts. This reveals pretraining and inference compute are not independent resources.
Research shows test-time scaling methods split into internal (training models for autonomous reasoning) and external (inference-time search and verification). They complement rather than compete; internal builds capability while external extracts performance from existing capability.
Research identifies internal vs external as the primary taxonomic split for test-time scaling, with training-side constraints (policy entropy collapse) and novel directions that shift *when* compute happens (sleep-time, post-completion) rather than just *how much*. Methods like consensus games and recursive LMs sidestep traditional scaling tradeoffs.
Reasoning models persistently outperform non-reasoning models regardless of inference budget because training instills a reasoning protocol that makes additional tokens productive. The gap is fundamentally about deployment mechanisms and training structure, not raw capability.
Augmenting pretraining data with LLM-generated reasoning traces improves data efficiency 3x and reasoning benchmark performance 10%+ for 3B models. Harder tokens automatically receive longer traces, creating a natural compute-allocation mechanism analogous to test-time scaling.
Show all 11 sources
Research shows that uniform inference budgets waste compute; allocation should vary by prompt. Test-time compute can substitute for training-time scaling on hard problems, but cannot overcome fundamental limitations set by the training regime.
Parallel methods improve coverage; sequential methods enable depth. The optimal choice depends on task structure: parallel wins for independent short problems, sequential for compositional chains requiring intermediate accumulation.
GRAM demonstrates that recursive reasoning models should maintain and explore multiple latent trajectories in parallel, not only deepen single paths. Width-scaling avoids the serial latency penalty of depth while sampling the solution distribution more effectively on ambiguous problems.
Information-theoretic analysis shows BoN and MCTS converge in reasoning accuracy when controlling for total compute. Snowball errors accumulate per step regardless of framework; mitigation depends on search scope and reward function reliability, not the specific algorithm.
Longer thinking traces improve accuracy through variance expansion—broader output distributions cover correct answers more often—not through better reasoning. Beyond a critical threshold, the distribution becomes too diffuse and accuracy drops, revealing the mechanism is sampling coverage, not genuine reasoning improvement.
Research shows that deep research agents exhibit test-time scaling laws where search steps scale similarly to reasoning tokens, and live search outperforms memorized retrieval on knowledge-intensive tasks. Data efficiency is extreme—78 curated demonstrations outperform 10K samples for agency.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Does Thinking More always Help? Understanding Test-Time Scaling in Reasoning Models
- When More Thinking Hurts: Overthinking in LLM Test-Time Compute Scaling
- Learning to Think: Information-Theoretic Reinforcement Fine-Tuning for LLMs
- RLAD: Training LLMs to Discover Abstractions for Solving Reasoning Problems
- Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters
- Think Twice: Enhancing LLM Reasoning by Scaling Multi-round Test-time Thinking
- Retrieval-augmented reasoning with lean language models
- A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?