What is the distinction between teaching reasoning how versus when to activate?
This explores a finding the corpus keeps circling back to: that a model's reasoning ability and its judgment about *when* to switch reasoning on are two separate things — one built during pre-training, the other tuned afterward.
This explores the difference between a model *having* reasoning skill and *knowing when to use it* — and the corpus draws a surprisingly sharp line between the two. The cleanest statement comes from work showing that reinforcement-learning post-training doesn't actually teach a model to reason; pre-training already planted that capability. What RL adds is timing — knowing when to deploy reasoning and when to answer directly. The evidence is striking: a hybrid that paired a base model's raw reasoning with a thinking model's steering recovered 91% of the performance gain using just 12% of the tokens Does RL teach reasoning or just when to use it?. That reframes RL as a deployment optimizer, not a capability creator — and it's the organizing idea behind proposals to architect reasoning systems that deliberately separate *activation timing* from *execution capability* How should reasoning systems actually be architected?.
The 'capability is already there' half of the story shows up everywhere once you look. A single latent feature, identified and steered directly, matches chain-of-thought performance without any prompting — implying the reasoning mode is a built-in switch waiting to be flipped Can we trigger reasoning without explicit chain-of-thought prompts?. Four modular 'cognitive tools' lifted GPT-4.1 on a hard math benchmark from 27% to 43% with no training at all, just by isolating operations the model could already perform Can modular cognitive tools unlock reasoning without training?. And critique fine-tuning unlocked reasoning from exposure to a *single* problem — suggesting what's needed is an activation signal, not new skill Can a single problem unlock reasoning through solution critique?. All three point the same way: the 'how' is latent; the work is in the 'when.'
But 'when' turns out to be subtle, because more reasoning is not always better. Vanilla models, left to think freely, talk themselves into self-doubt that *degrades* answers — and RL training's real job is redirecting that same thinking machinery from spiraling into productive gap-analysis Does extended thinking help or hurt model reasoning?. Instance-adaptive analysis shows simple questions do better with a direct question-to-answer path than with step-by-step reasoning, so the optimal move depends on the specific question, not the task category Why do some questions perform better without step-by-step reasoning?. Knowing 'when' includes knowing *when not to* — and getting that wrong has costs: scaling reasoning capability quietly erodes instruction-following, because longer chains pull attention away from the original request Why do better reasoning models ignore instructions?.
The one place the corpus pushes back on the clean split is *where* the 'how' gets built. If capability really is fixed in pre-training, you might plant it more deliberately there: treating chain-of-thought as an exploratory action rewarded by information gain lifts reasoning ~19% during pre-training itself Can chain-of-thought reasoning be learned during pretraining itself?. And the 'how' may be structural — reasoning generalizes from broad procedural knowledge picked up across many documents, unlike facts, which are memorized narrowly Does procedural knowledge drive reasoning more than factual retrieval? — which even maps onto network anatomy, with knowledge concentrated in lower layers and reasoning in higher ones Why does reasoning training help math but hurt medical tasks?.
The thing you didn't know you wanted to know: the recent leap in 'reasoning models' may be less about teaching machines to think and more about teaching them *restraint* — when to think, how hard, and when to just answer. The skill was already in the box; what was missing was the judgment to use it.
Sources 11 notes
Pre-training acquires reasoning capability; RL teaches efficient deployment. A hybrid model combining base reasoning with thinking model steering recovered 91% of performance gains using only 12% of tokens, suggesting RL acts as a deployment optimizer rather than a capability creator.
Research shows RL post-training teaches models *when* to use reasoning mechanisms that pre-training already provides. Decoupled architectures, latent reasoning in continuous space, and interleaved action-grounding all outperform monolithic chain-of-thought approaches.
SAE-identified reasoning features can be directly steered to match or exceed chain-of-thought performance across six model families. This reasoning mode activates early in generation and overrides surface-level instructions, suggesting latent reasoning is a fundamental capability independent of explicit prompting.
Four cognitive tools implemented as sandboxed LLM calls improved GPT-4.1 on AIME2024 from 26.7% to 43.3% without any RL training. Modularity enforces operation isolation that pure prompting cannot guarantee, eliciting pre-existing reasoning capability.
Critique Fine-Tuning achieves reasoning activation comparable to RLVR using only one problem and teacher-generated critiques of varied solutions, with no reinforcement learning. This demonstrates that exposure to correct versus incorrect reasoning on a specific problem is the sufficient activation signal.
Vanilla models use thinking mode counterproductively, inducing self-doubt that degrades performance. RL training reverses this, transforming the same mechanism into beneficial gap analysis. Training mediates reasoning quality, not just quantity.
Saliency analysis reveals that CoT prompting fails when question information doesn't aggregate into the prompt structure before reasoning begins. For simple questions, direct question-to-answer flow outperforms step-by-step reasoning, showing the optimal prompt depends on question type, not just task category.
The MathIF benchmark shows that SFT and RL training improve reasoning but reduce instruction adherence, particularly as chain-of-thought length increases. Longer reasoning chains create contextual distance that dilutes the model's attention to original instructions.
RLP treats CoT as exploratory action during pretraining, using log-likelihood improvement as verifier-free reward. Applied to Qwen3-1.7B and Nemotron-Nano-12B, the method improves math and science benchmarks substantially, suggesting reasoning can be planted earlier in training.
Analysis of 5 million pretraining documents shows reasoning relies on broad, transferable procedural knowledge from diverse sources, unlike factual recall which depends on narrow, document-specific memorization of target facts.
Two-phase inference model shows knowledge retrieval operates in lower network layers while reasoning adjustment happens in higher layers. This separation explains why reasoning training improves math but can degrade knowledge-intensive domains like medicine.