What training signals would teach models when not to reason?
This explores what kinds of training rewards or objectives would teach a model the skill of disengaging — knowing when a question is unanswerable, when to stay silent, when to abstain, or when to answer directly instead of grinding through reasoning steps.
This explores what training signals would teach a model the inverse of reasoning — when *not* to spin up a chain of thought, when to say "I don't know," when to stay quiet, or when to flag that a question can't be answered at all. The corpus has a surprisingly coherent answer hiding across several notes: the reason models don't know when to stop is that almost every standard training signal rewards them for producing more. The fix isn't better reasoning — it's a different reward shape entirely.
Start with the diagnosis, because it's sharper than you'd expect. Reasoning fine-tuning actively *degrades* a model's ability to decline — abstention capacity drops about 24% because the training signal rewards complete answers and punishes "I don't know" Does reasoning fine-tuning make models worse at declining to answer?. The same pressure shows up when a question is broken: reasoning models will generate long, redundant traces for questions with missing premises, while plainer models correctly flag them as unanswerable — because the objective optimizes for producing steps and never teaches disengagement Why do reasoning models overthink ill-posed questions?. And scaling reasoning quietly costs instruction-following: longer chains create "contextual distance" that dilutes attention to the original ask Why do better reasoning models ignore instructions?. So the question "what teaches a model not to reason" is partly "what undoes the reflex that reasoning training installs."
The most direct answers in the corpus reframe restraint as an explicit decision the model must learn to make, rather than a behavior we hope emerges. Thinkless trains a single model to *route* between extended reasoning and a direct response, using a decoupled RL scheme (DeGRPO) that separates the mode-selection signal from the answer-quality signal — without that decoupling, the model collapses into one mode Can models learn when to think versus respond quickly?. DiscussLLM does the conversational analog: it makes "stay silent" a first-class classification outcome alongside speaking, training timing as an objective in its own right rather than a side effect Can models learn when NOT to speak in conversations?. The shared move is the lesson: restraint has to be its own reward target, decoupled from the reward for being helpful or correct, or the helpfulness signal swamps it.
The deeper signal, though, is calibration. A model that knows *when* not to reason is really a model that knows when it doesn't know. Small models trained with uncertainty-aware objectives and an explicit abstention option match models 10x their size on forecasting — which tells you calibration ability already exists in these networks but sits undertrained under standard recipes Can models learn to abstain when uncertain about predictions?. That connects to a broader finding: base models already contain latent reasoning that minimal signals merely *select* rather than create Do base models already contain hidden reasoning ability?. If reasoning is selected, not built, then restraint can probably be selected too — the capacity to abstain may be sitting there, suppressed by reward shapes that pay only for answers.
Two cautions worth carrying. First, don't expect restraint to fall out of "more reasoning, but better" — sycophancy research shows reasoning-optimized models resist social pressure no better than base ones, because the failure lives in the generation distribution, not the reasoning process Can better reasoning training actually reduce model sycophancy?. Restraint is the same kind of problem: a distribution that needs reshaping, not a reasoning chain that needs lengthening. Second, there's a clue about *what* the signal can be cheap to provide — corrupted reasoning traces teach about as well as correct ones, implying the trace is computational scaffolding rather than meaningful content Do reasoning traces need to be semantically correct?. If correctness of the trace barely matters, then the signal worth investing in isn't "reason well" but "decide whether to reason at all" — which is exactly the routing, silence, and abstention objectives above, and the reason multi-turn reward design (rewarding long-term interaction value over immediate helpfulness) keeps surfacing as the lever that lets models hold back and ask instead of charging ahead Why do language models respond passively instead of asking clarifying questions?.
Sources 10 notes
Models optimized for reasoning performance answer questions more often but express unwarranted confidence and fail to abstain appropriately. The training signal rewards complete answers, systematically punishing 'I don't know' responses.
Reasoning models generate redundant, lengthy responses to questions with missing premises while non-reasoning models correctly identify them as unanswerable. Training optimizes for producing reasoning steps but never teaches models when to disengage.
The MathIF benchmark shows that SFT and RL training improve reasoning but reduce instruction adherence, particularly as chain-of-thought length increases. Longer reasoning chains create contextual distance that dilutes the model's attention to original instructions.
Thinkless trains a single model to select between extended reasoning and direct responses using DeGRPO, which decouples mode selection from answer refinement. This prevents mode collapse and enables self-calibrated routing without explicit difficulty labels.
DiscussLLM trains AI to decide between five intervention types or remaining silent using an 88K synthetic discussion dataset. A decoupled classifier-generator architecture achieves better computational efficiency, while end-to-end training better integrates when-to-speak and what-to-say decisions.
Small open-source models trained with uncertainty-aware objectives and abstention capabilities match 10x larger pre-trained models on conversation forecasting. This shows calibration ability exists but remains undertrained in standard LLMs.
Five independent mechanisms—RL steering, critique fine-tuning, decoding changes, SAE feature steering, and RLVR—all elicit reasoning already present in base model activations. Post-training selects rather than creates reasoning; the bottleneck is elicitation, not capability acquisition.
Reasoning-optimized models show no meaningful resistance advantage to sycophantic pressure compared to base models. The LOGICOM benchmark found GPT-4 still fell for logical fallacies 69% more often, suggesting sycophancy is a generation-distribution problem, not a reasoning problem.
Models trained on systematically irrelevant traces maintain solution accuracy and sometimes improve out-of-distribution generalization, suggesting traces function as computational scaffolding rather than meaningful reasoning steps.
CollabLLM demonstrates that standard RLHF training optimizes for immediate helpfulness, discouraging models from asking clarifying questions or offering multi-turn insights. Multi-turn-aware rewards that estimate long-term interaction value enable active intent discovery and genuine collaboration.