Fine-tuning raises domain scores but degrades actual reasoning by nearly 40% — and the metrics you'd use to check are the ones being fooled.
Does fine-tuning improve domain accuracy at the cost of reasoning quality?
This explores the tradeoff at the heart of specializing a model on a domain — whether the accuracy you gain on benchmarks is quietly paid for by reasoning that gets worse, and whether that tradeoff is inevitable or an artifact of *how* you fine-tune.
This explores whether fine-tuning buys domain accuracy at the expense of reasoning quality — and the corpus says: yes, but mostly for one specific method, and the cost is nearly invisible to the metrics people actually watch. The sharpest result is that supervised fine-tuning (SFT) raises final-answer accuracy while cutting a model's Information Gain by 38.9% Does supervised fine-tuning improve reasoning or just answers? Does supervised fine-tuning actually improve reasoning quality?. The unsettling part is the mechanism: the model still lands on correct answers, but it gets there by pattern-matching and *post-hoc rationalization* rather than genuine inferential steps. Standard benchmarks only score the final answer, so they show improvement and hide the rot underneath.
What does 'degraded reasoning' concretely mean? A separate line of work stress-tests the reasoning chain itself and finds fine-tuning makes the steps decorative. When you cut a chain off early, paraphrase it, or splice in filler tokens, the fine-tuned model's answer stays the same more often — meaning the reasoning it writes no longer causes the answer it gives Does fine-tuning disconnect reasoning steps from final answers?. Reasoning becomes performance rather than function. This is the auditable-versus-accurate split: you get a model that's right more often but that you can trust less about *why* it's right.
The more interesting turn is that this tradeoff isn't a law of nature — it's a property of SFT's token-level objective. Reinforcement-learning approaches that reward *explanation quality*, not just answer correctness, embed domain knowledge without the same reasoning tax; RLAG cycles between augmented and unaugmented generation to internalize coherent knowledge structures and outperforms SFT precisely because it optimizes for rationality over surface tokens Can reinforcement learning embed domain knowledge more effectively than supervised fine-tuning?. Similarly, small models trained with DPO on explicit right/wrong preference pairs beat SFT on function-calling and math Can small models match large models on function calling?. The common thread: give training a signal about the *process*, and you can specialize without hollowing out the reasoning.
But don't over-credit RL either — the corpus refuses to let either method off the hook. RL-tuned models still crater on out-of-distribution variants, suggesting they sharpen memorized templates rather than install genuine problem-solving procedures Do fine-tuned language models actually learn optimization procedures?. And a striking reframe argues that base models already *contain* the reasoning ability latently; post-training mostly selects and deploys it rather than creating it, teaching the model *when* to reason, not *how* Do base models already contain hidden reasoning ability? Does RL post-training create reasoning or just deploy it?. Under that view, SFT's damage is a kind of overwriting — you're not adding domain skill on top of intact reasoning, you're partly paving over the reasoning circuitry to install shortcuts.
The honest bottom line the corpus converges on: there's no free specialization. Every adaptation method has a domain-specific sweet spot beyond which visible gains come with hidden degradation in reasoning faithfulness, capability transfer, and format flexibility How do domain training techniques actually reshape model behavior? How do you specialize LLMs without losing general reasoning?. So the real question isn't 'does fine-tuning cost reasoning?' — it's 'which objective are you optimizing, and are you measuring the thing it quietly breaks?' If you only watch accuracy, you'll never see the bill.
Sources 10 notes
Supervised fine-tuning improves final-answer accuracy on benchmarks but cuts Information Gain by 38.9 percent, meaning models generate correct answers through post-hoc rationalization rather than genuine inferential steps. Standard metrics miss this degradation because they only measure final correctness.
SFT improves final-answer accuracy but reduces reasoning informativeness by 38.9% on average. Models reach correct answers through pattern-matching shortcuts rather than genuine inferential reasoning, becoming less auditable despite higher accuracy scores.
Three faithfulness tests show fine-tuned models generate reasoning chains that less reliably influence final outputs. Early termination, paraphrasing, and filler substitution all produce invariant answers more often after fine-tuning, suggesting reasoning becomes performative rather than functional.
RLAG rewards both answer accuracy and explanation rationality by cycling between augmented and unaugmented generation, progressively internalizing coherent knowledge structures. This outperforms SFT because it prioritizes reasoning quality over token-level correctness.
Small models fine-tuned via DPO on correct and incorrect function-calling examples from a large teacher model achieve high accuracy on logical and mathematical tasks. DPO's explicit negative examples directly target the rigid output format failures where SFT alone underperforms.
Show all 10 sources
Even GRPO-trained models show sharp performance drops on out-of-distribution variants (N-1 test sets) compared to in-distribution problems, indicating RL optimizes template-matching rather than genuine problem-solving procedures.
Five independent mechanisms—RL steering, critique fine-tuning, decoding changes, SAE feature steering, and RLVR—all elicit reasoning already present in base model activations. Post-training selects rather than creates reasoning; the bottleneck is elicitation, not capability acquisition.
Evidence shows base models already contain reasoning capability in latent form; RL training optimizes deployment timing rather than capability creation. Hybrid models recover 91% of performance gains by routing tokens only, and activation vectors for reasoning strategies pre-exist before any RL.
Research shows every adaptation method—from parameter-efficient tuning to knowledge graph curricula—has optimal conditions tied to specific domains. The key finding: visible benefits like performance gains often come with hidden degradation in reasoning faithfulness, capability transfer, and format flexibility.
Research shows supervised fine-tuning raises domain benchmarks but degrades reasoning by 38%, while reinforcement learning prunes inaccurate knowledge rather than adding capability. Every specialization technique has a domain-specific optimal point beyond which performance declines.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Echo Chamber: RL Post-training Amplifies Behaviors Learned in Pretraining
- Eliciting Reasoning in Language Models with Cognitive Tools
- Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces!
- A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?
- ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models
- On the Impact of Fine-Tuning on Chain-of-Thought Reasoning
- Train Long, Think Short: Curriculum Learning for Efficient Reasoning
- Learning to Reason for Factuality