If reasoning is already baked into the model and any of several training tweaks can surface it, no single tweak gets sole credit.
Why do reasoning gains resist clear attribution to specific training changes?
This explores why it's so hard to point at one training change and say *that's* what improved reasoning — and the corpus suggests the reason is that training mostly selects reasoning the base model already had, through a tangle of interacting factors.
This explores why reasoning gains resist clean attribution — why you can rarely say "this tweak caused that improvement." The corpus points to a deeper reason than measurement noise: much of the time, training isn't *creating* reasoning at all, it's *selecting* it. Five independent interventions — RL steering, critique fine-tuning, decoding changes, feature steering, and RLVR — all elicit reasoning that already sits latent in the base model's activations Do base models already contain hidden reasoning ability?. If the capability is already there and many different levers unlock the same thing, then no single lever "owns" the gain. The bottleneck is elicitation, not acquisition, which is exactly the situation where attribution dissolves.
Even when you try to isolate the ingredient responsible, the ingredient turns out not to be isolable. The reusable unit of post-training isn't a dataset or a prompt-response pair — it's a *feedback interface* entangled with six co-dependent factors: the verifier, the base model, the data lineage, the optimizer, the scaffold, and the compute budget What is the actual reusable unit of reasoning data?. Change any one and the *same* data produces a different effect. Attribution only becomes tractable when all six are reported together, which they rarely are. So a "training change" that looks discrete on paper is actually a move through a six-dimensional space where the axes interact.
Part of the confusion is that the gains often trace back to *pretraining*, not the post-training step people are trying to credit. Reasoning generalization is driven by broad procedural knowledge distributed across many pretraining documents — how-to patterns — rather than by narrow factual recall Does procedural knowledge drive reasoning more than factual retrieval?. And CoT-style reasoning can be planted during pretraining itself using information-gain rewards, lifting downstream benchmarks well before any fine-tuning happens Can chain-of-thought reasoning be learned during pretraining itself?. If the seed was sown early and diffusely, the late-stage tweak you're measuring is drawing on a reservoir you didn't create and can't fully see.
Attribution gets even slipperier because the visible improvement and the real improvement can come apart. Imitation training makes models *sound* better — confident, fluent, ChatGPT-like — while closing no actual capability gap; evaluators are fooled by style Can imitating ChatGPT fool evaluators into thinking models improved?. Fine-tuning can make chain-of-thought *less* causally connected to the final answer, so the reasoning becomes performative rather than load-bearing even as scores hold Does fine-tuning disconnect reasoning steps from final answers?. And CoT itself is distribution-bounded — fluent but logically hollow once you push past the training distribution Does chain-of-thought reasoning actually generalize beyond training data?. A metric can move for reasons that have nothing to do with better reasoning.
Finally, some gains emerge from reward *dynamics* rather than any explicit instruction, which further scrambles cause and effect. Models trained with RL drift toward shorter chains as they get more capable — simplicity emerges from the reward signal, not from anyone training brevity in Why does chain of thought accuracy eventually decline with length? — and accuracy follows an inverted-U in thinking length, peaking then degrading past a token threshold Does more thinking time always improve reasoning accuracy?. Add that reasoning lives in higher network layers while knowledge sits lower, so the same reasoning push helps math yet quietly degrades medicine Why does reasoning training help math but hurt medical tasks? — and you get the real lesson: reasoning gains resist attribution because they're an emergent property of a coupled system — latent capability, pretraining priors, entangled feedback interfaces, and reward dynamics — not the output of a switch you flipped.
Sources 10 notes
Five independent mechanisms—RL steering, critique fine-tuning, decoding changes, SAE feature steering, and RLVR—all elicit reasoning already present in base model activations. Post-training selects rather than creates reasoning; the bottleneck is elicitation, not capability acquisition.
The reusable unit in post-training is a feedback interface entangled with six factors: verifier, base model, lineage, optimizer, scaffold, and budget. Changing any one alters the same data's effect, making attribution tractable only when these are jointly released.
Analysis of 5 million pretraining documents shows reasoning relies on broad, transferable procedural knowledge from diverse sources, unlike factual recall which depends on narrow, document-specific memorization of target facts.
RLP treats CoT as exploratory action during pretraining, using log-likelihood improvement as verifier-free reward. Applied to Qwen3-1.7B and Nemotron-Nano-12B, the method improves math and science benchmarks substantially, suggesting reasoning can be planted earlier in training.
Imitation models fool human evaluators by mimicking ChatGPT's confident, fluent style while failing to improve factuality or generalization on novel tasks. The ceiling is set by base model capability, not fine-tuning method—better fundamentals, not shortcuts, drive real improvement.
Show all 10 sources
Three faithfulness tests show fine-tuned models generate reasoning chains that less reliably influence final outputs. Early termination, paraphrasing, and filler substitution all produce invariant answers more often after fine-tuning, suggesting reasoning becomes performative rather than functional.
DataAlchemy experiments show CoT fails systematically under distributional shifts in task, length, and format. Models produce fluent but logically inconsistent reasoning — imitating reasoning form without valid underlying logic.
Task accuracy peaks at intermediate CoT length, with optimal length increasing alongside task difficulty but decreasing with model capability. RL training naturally gravitates toward shorter chains as models improve, revealing that simplicity emerges from reward signals rather than explicit training.
Increasing thinking tokens from ~1,100 to ~16K reduced benchmark accuracy from 87.3% to 70.3%, revealing a non-monotonic relationship where models overthink easy problems and underthink hard ones.
Two-phase inference model shows knowledge retrieval operates in lower network layers while reasoning adjustment happens in higher layers. This separation explains why reasoning training improves math but can degrade knowledge-intensive domains like medicine.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Eliciting Reasoning in Language Models with Cognitive Tools
- On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Models
- Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens
- Rethinking Thinking Tokens: LLMs as Improvement Operators
- Mining Hidden Thoughts from Texts: Evaluating Continual Pretraining with Synthetic Data for LLM Reasoning
- An Enigma of Artificial Reason: Investigating the Production-Evaluation Gap in Large Reasoning Models
- When More is Less: Understanding Chain-of-Thought Length in LLMs
- Base Models Know How to Reason, Thinking Models Learn When