Teaching a model to reason while it trains — not just when it answers — can cut the data it needs by 3x.

This explores whether inference-time compute improves pretraining data efficiency — and the most direct answer in the corpus flips the question: the biggest efficiency gains come from importing the *logic* of test-time scaling back into pretraining itself. The clearest evidence is thinking-augmented pretraining, where pretraining data is enriched with LLM-generated reasoning traces and harder tokens automatically attract longer traces — a built-in compute-allocation mechanism that mirrors how test-time scaling spends more on hard prompts. The result is a 3x data-efficiency gain and a 10%+ reasoning bump for a 3B model Can training data augmentation match test-time compute scaling benefits?. So in practice, the answer isn't "add inference compute to a fixed model" — it's "teach the model to reason while pretraining, and you need less data."

That reframing matters because raw inference compute alone hits a wall. Non-reasoning models don't catch up to reasoning models no matter how large the inference budget, because what makes extra tokens *productive* is a reasoning protocol instilled during training, not the tokens themselves Can non-reasoning models catch up with more compute?. Inference compute is a lever only when training has already shaped the model to use it well. The complementary move is to make that protocol part of pretraining from the start: RLP treats chain-of-thought as an exploratory action during pretraining and rewards it by how much it improves next-token prediction, lifting math and science benchmarks ~19% — reasoning planted earlier rather than bolted on after Can chain-of-thought reasoning be learned during pretraining itself?.

There's a deeper reason this works so well, and it's the thing you might not expect: a lot of "new" reasoning capability was already latent in the base model. Five independent methods — RL steering, critique fine-tuning, decoding tweaks, SAE feature steering, RLVR — all surface reasoning that already lives in base-model activations, meaning post-training *selects* rather than *creates* the ability Do base models already contain hidden reasoning ability?. If the capability is mostly there, then the efficiency question becomes one of elicitation, and inference-time or lightweight-training methods are cheap ways to unlock what your pretraining data already paid for. Adaptive test-time compute fits here too: spending more on hard prompts and less on easy ones beats uniform budgets How should we spend compute at inference time? — useful, but it's harvesting latent capability, not manufacturing new data efficiency.

The honest caveat is that inference-time reasoning doesn't generalize for free. Chain-of-thought degrades predictably once you push outside the training distribution — models produce fluent but logically broken reasoning, imitating the *form* of reasoning without the underlying logic Does chain-of-thought reasoning actually generalize beyond training data?. So inference compute amplifies whatever the training distribution taught; if the data didn't cover a regime, more thinking tokens won't conjure competence there. This is exactly why the corpus leans toward building reasoning into pretraining and toward learning *when* to think versus answer directly Can models learn when to think versus respond quickly? — you get the efficiency by allocating compute intelligently across both phases, not by treating inference as a substitute for data.

The takeaway a curious reader walks away with: "inference-time compute improves data efficiency" is true mainly in an indirect, surprising way — the winning recipe is to *move test-time reasoning into pretraining* (generate the thinking, reward it during training), because the base model already holds latent reasoning and what's scarce is the protocol that makes extra compute pay off.