Can an AI just think longer to make up for how it was trained, or does training set the ceiling?
Does more inference compute help close gaps between different training regimes?
This explores whether spending more compute at inference time (longer thinking, more passes, more samples) can make up for a model having been trained differently, or whether the training regime sets a ceiling that extra compute can't lift.
This explores whether extra inference compute can make up for a different training regime. For the clearest case, the corpus says no. Reasoning models keep outperforming non-reasoning models at any inference budget, because training instills a reasoning protocol that makes additional tokens productive Can non-reasoning models catch up with more compute?. Extra compute multiplies what training set up. It doesn't replace it.
The reason is that the gap is less about capability than about access. Five different methods (RL steering, critique fine-tuning, decoding changes, SAE feature steering and RLVR) all pull reasoning out of base models that already contained it Do base models already contain hidden reasoning ability?. A training regime mostly decides whether a model knows how to spend its tokens. That changes what closes gaps at inference time: better elicitation, not more volume. Proxy-tuning closes 88-91% of the alignment gap by shifting output distributions at decoding time, with the base weights untouched Can decoding-time tuning preserve knowledge better than weight fine-tuning?. A stronger model nearly doubled a weaker model's Theory-of-Mind scores by building it a harness. That harness worked mainly by moving unstable reasoning into deterministic code and task-specific routing, not by encouraging longer thinking Can a stronger model lift a weaker one at test time without retraining?. Inference-time structure can bridge training gaps where inference-time volume can't.
Looped models are the exception that clarifies the rule. They spend more compute by re-applying the same layers, and they beat larger feedforward networks on reasoning Can models learn by looping instead of growing larger?. But they get there because training anticipated the loops. Ouro's 1.4B-2.6B models match 12B baselines because looping is built into pretraining Can reasoning be learned during pretraining rather than after?. LOTUS closes the latent-versus-explicit chain-of-thought gap at 3B scale only because gold reasoning supervision is applied at each latent position Can latent reasoning close the scaling gap with explicit chain-of-thought?. Sparse attention shows the same pattern from another angle. It matches dense attention at 109B because its block selector was trained natively, not retrofitted Can sparse attention match dense models without retrofitting?. In each case compute pays off when training and inference agree on how it will be spent.
Training also shapes what the extra tokens look like, and many of them may be dispensable. RL post-training collapses a model onto one dominant pretraining format and suppresses the alternatives Does RL training collapse format diversity in pretrained models?. So part of the gap between regimes is which style of thinking got amplified, and a longer budget doesn't recover the style that was suppressed. Models can also be trained to decide when extended thinking is worth it at all Can models learn when to think versus respond quickly?. Verbosity turns out to be a single steerable direction in activation space, and pushing along it cut chain-of-thought length by 67% with accuracy maintained Can we steer reasoning toward brevity without retraining?. That suggests much of a long reasoning trace is slack.
The question worth asking is whether a model was trained to turn compute into answers, not how much compute it gets. Even efficiency is partly a training-time decision. Architectures tuned with conditional scaling laws delivered 42% more throughput and 2.1% higher accuracy than LLaMA-3.2 under the same training budget Can architecture choices improve inference efficiency without sacrificing accuracy?. More inference compute closes a gap only when training prepared the model to use it.
Sources 12 notes
Reasoning models persistently outperform non-reasoning models regardless of inference budget because training instills a reasoning protocol that makes additional tokens productive. The gap is fundamentally about deployment mechanisms and training structure, not raw capability.
Five independent mechanisms—RL steering, critique fine-tuning, decoding changes, SAE feature steering, and RLVR—all elicit reasoning already present in base model activations. Post-training selects rather than creates reasoning; the bottleneck is elicitation, not capability acquisition.
Proxy-tuning closes 88-91% of the alignment gap while surpassing direct fine-tuning on knowledge tasks by leaving base model weights untouched. Direct fine-tuning corrupts knowledge storage in lower layers, whereas proxy-tuning applies distributional shifts that primarily affect reasoning and style.
A stronger model built inference-time harnesses that nearly doubled weaker model performance on Theory-of-Mind benchmarks without retraining, primarily by moving unstable reasoning into deterministic code and task-specific routing rather than encouraging extended reasoning.
Models that re-apply layers in recurrent depth outperform larger feedforward networks on reasoning tasks. This works because recursion enables state tracking and compositional generalization that parameter scaling alone cannot achieve, with convergence signals providing natural halting.
Show all 12 sources
Ouro's 1.4B–2.6B models match 12B baselines by performing reasoning during pretraining via iterative latent loops, not by storing more knowledge. Their intermediate latent states align strongly with final outputs, making them more faithful than divergent chain-of-thought traces.
LOTUS, a looped Transformer trained with gold CoT supervision at each latent position, closes the latent-to-explicit gap on GSM8K and cuts thought-phase latency by 2.5× to 6.9×. Latent reasoning need not be verbalized token-by-token to remain legible and competitive.
MiniMax Sparse Attention achieves 28.4× per-token attention compute reduction at 1M context while matching full-attention GQA performance at 109B, because its block selector is trained end-to-end during native pretraining rather than retrofitted. This proves sparsity can be a frontier move, not just an efficiency patch.
Controlled experiments show RL consistently amplifies one format distribution from pretraining within the first epoch while collapsing alternatives. The winning format depends on model scale, not necessarily performance, and is largely hidden when starting from proprietary pretrained models.
Thinkless trains a single model to select between extended reasoning and direct responses using DeGRPO, which decouples mode selection from answer refinement. This prevents mode collapse and enables self-calibrated routing without explicit difficulty labels.
Activation-Steered Compression extracts a single vector from 50 paired examples to reduce chain-of-thought length by 67% while maintaining accuracy and achieving 2.73x speedup. The method is training-free and generalizes across model sizes and domains.
Augmenting scaling laws with hidden size, MLP-to-attention ratio, and GQA configuration enables architecture optimization for inference. Optimized models achieved up to 2.1% higher accuracy and 42% greater throughput than LLaMA-3.2 under identical training budgets.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Evaluating Theory of Mind in Reasoning Models: Robustness over Reasoning
- Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach
- Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers
- Base Models Know How to Reason, Thinking Models Learn When
- Reasoning Beyond Chain-of-Thought: A Latent Computational Mode in Large Language Models
- The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity
- Echo Chamber: RL Post-training Amplifies Behaviors Learned in Pretraining
- Hierarchical Reasoning Model