Does fine-tuning a small model match fine-tuning a large one?
This explores whether a small model, once fine-tuned, can close the gap with a fine-tuned large model — and the corpus suggests the answer depends entirely on what you're asking the model to do.
This is really asking whether size still matters after you've fine-tuned — and the collection's most striking finding is that for many tasks, it doesn't. Small models fine-tuned with DPO on a large teacher's correct-and-incorrect function-calling examples reach high accuracy on logical and mathematical tasks, with the explicit negative examples patching exactly the rigid-format failures that plain supervised fine-tuning leaves behind Can small models match large models on function calling?. So 'match' is achievable — but the path matters: how you fine-tune (preference pairs vs. imitation) can matter more than how big the model is.
The deeper reason small can rival large is that scaling pretraining and scaling fine-tuning do different jobs. Pretraining scale buys factual knowledge stored in lower layers; fine-tuning scale buys behavioral helpfulness expressed in upper layers Do pretraining and fine-tuning scale independently in language models?. That decoupling explains the pattern: if your task is about behavior and format — calling tools correctly, answering helpfully — a small model's upper layers can be tuned to par. If your task leans on stored world-knowledge, the small model's smaller pretrained base is the real bottleneck, and fine-tuning won't manufacture facts it never learned.
There's an even more surprising angle: sometimes smaller wins outright. For generating diverse synthetic data, models around 500M parameters produce more unique outputs per sample than larger ones, which concentrate probability mass on their favorite answers Why aren't bigger models better for generating diverse outputs?. And on hard prompts, a small model given more inference-time compute can match a much larger one — parameters and 'thinking time' trade off against each other Can inference compute replace scaling up model size?. So the honest comparison isn't small-vs-large in isolation; it's small-plus-a-clever-budget vs. large.
But the corpus also warns that 'matching' can be a mirage, for small and large alike. Supervised fine-tuning often improves how outputs *look* — valid JSON, proper sections — without making them actually feasible or correct Does supervised fine-tuning actually improve reasoning on optimization problems?. RL fine-tuning can sharpen template-matching rather than install real reasoning procedures, with sharp drops on out-of-distribution variants Do fine-tuned language models actually learn optimization procedures?, and it can even loosen the causal link between a model's reasoning steps and its final answer Does fine-tuning disconnect reasoning steps from final answers?. So a fine-tuned small model that 'matches' on a benchmark may be matching the same surface competence — and the same brittleness.
The sharpest reframe is that on some problems, scale never mattered to begin with. On genuine constrained-optimization tasks, models plateau at roughly 55–60% constraint satisfaction regardless of parameter count, architecture, or training regime — a ceiling, not a scaling gap Do larger language models solve constrained optimization better?. Where a hard ceiling exists, a fine-tuned small model matches a fine-tuned large one trivially: both are stuck at the same wall. So the most useful version of your question may be: *what kind* of task are you fine-tuning for? For behavior and format, small can catch up; for stored knowledge, it usually can't; and for problems with a built-in ceiling, the whole comparison dissolves.
Sources 8 notes
Small models fine-tuned via DPO on correct and incorrect function-calling examples from a large teacher model achieve high accuracy on logical and mathematical tasks. DPO's explicit negative examples directly target the rigid output format failures where SFT alone underperforms.
Emulated Fine-Tuning reveals that scaling pretraining improves factual knowledge while scaling fine-tuning improves behavioral helpfulness. This decoupling has architectural roots: pretraining enriches lower-layer knowledge storage, while fine-tuning modifies upper-layer behavior expression.
Research shows that for synthetic data generation, models around 500M parameters outperform larger ones in output diversity per sample. Larger models concentrate probability mass on preferred outputs, reducing the variety of distinct samples generated within a fixed budget.
Snell et al. (2024) showed that inference-time compute trades off against model parameter scaling, especially on difficult prompts. This reveals pretraining and inference compute are not independent resources.
Supervised fine-tuning makes model outputs look correct—proper JSON structure, valid identifiers, expected sections—without making them physically feasible. The model learns surface features of solutions, not the reasoning to construct valid ones.
Even GRPO-trained models show sharp performance drops on out-of-distribution variants (N-1 test sets) compared to in-distribution problems, indicating RL optimizes template-matching rather than genuine problem-solving procedures.
Three faithfulness tests show fine-tuned models generate reasoning chains that less reliably influence final outputs. Early termination, paraphrasing, and filler substitution all produce invariant answers more often after fine-tuning, suggesting reasoning becomes performative rather than functional.
Across constrained-optimization tasks, LLMs converge to ~55–60% constraint satisfaction independent of architecture, parameter count, or training regime. Reasoning models do not systematically outperform standard models, suggesting a fundamental ceiling rather than a scaling gap.