AI learns reasoning from millions of documents, but each fact is memorized from just one — which makes fine-tuning surprisingly fragile.

This explores how models pick up two different kinds of knowledge — the how-to of reasoning (procedural) versus the recall of specific facts — and why pretraining and fine-tuning treat those two so differently. The cleanest split comes from tracing where each kind of knowledge actually lives. When a model reasons, it draws on broad, transferable procedures learned from many diverse documents during pretraining; when it recalls a fact, it leans on narrow memorization of the specific document that carried that fact Does procedural knowledge drive reasoning more than factual retrieval?. So the two aren't just different skills — they're stored and sourced differently from the very start. Procedure generalizes because it was distilled across sources; facts stay brittle because each one was memorized in one place.

That storage difference is what makes fine-tuning risky. Because facts sit in the lower, memory-oriented layers of the network, direct weight fine-tuning tends to corrupt exactly the regions where pretrained knowledge is stored — whereas leaving the weights alone and nudging the model at decoding time preserves that knowledge while still shifting reasoning and style Can decoding-time tuning preserve knowledge better than weight fine-tuning?. Even when fine-tuning successfully injects a new fact, the model often can't use it: the fact gets routed into storage layers and never reaches the mid-layer circuits that do reasoning, so the model 'knows' it but can't deploy it. Relocating those representations recovers most of the lost generalization, which shows the problem is a routing failure, not a capacity limit Why do fine-tuned models fail to use facts they memorize?.

Here's the part that reframes the whole question: much of what we call 'learning to reason' during fine-tuning isn't learning at all. Base models already carry latent reasoning capability, and a handful of independent methods — RL steering, critique tuning, decoding tricks, feature steering — all just elicit what's already there rather than create it Do base models already contain hidden reasoning ability?. Post-training seems to teach *when* to reason, not *how*; hybrid models recover most of the gains by only routing which tokens get the reasoning treatment Does RL post-training create reasoning or just deploy it?. Instruction tuning tells a similar story — models trained on semantically empty or even wrong instructions match those trained on correct ones, because what transfers is the output format, not task understanding Does instruction tuning teach task understanding or output format?.

This is why supervised fine-tuning can quietly hollow out procedure while looking like progress. SFT raises benchmark accuracy but cuts the quality of the reasoning steps by nearly 39% — models learn to produce the right answer through post-hoc rationalization instead of genuine inference Does supervised fine-tuning improve reasoning or just answers?. The reasoning chain also becomes performative: after fine-tuning, the intermediate steps causally influence the final answer far less, so the 'procedure' is decorative rather than functional Does fine-tuning disconnect reasoning steps from final answers?. Optimizing for factual correctness at the token level actively degrades the transferable procedural machinery.

The corpus points toward a resolution: match the training method to the knowledge type. RL that rewards explanation quality — not just answer correctness — internalizes coherent knowledge structures better than SFT because it prioritizes reasoning over token-matching Can reinforcement learning embed domain knowledge more effectively than supervised fine-tuning?. And the most radical move is to stop treating reasoning as a fine-tuning afterthought: looped architectures build procedure directly into pretraining through iterative latent computation, letting small models match far larger ones not by storing more facts but by reasoning better during pretraining itself Can reasoning be learned during pretraining rather than after?. The through-line: pretraining is where durable procedure gets built, and fine-tuning is far better at reshaping facts and formats than at teaching genuinely new ways to think.