SYNTHESIS NOTE
Topics›Tool Computer Use›this note

Can small models match large models on function calling?

Explores whether small language models fine-tuned with the right training method can achieve comparable performance to large models on structured reasoning tasks requiring precise function calls, and what training approach makes this possible.

Synthesis note · 2026-05-03 · sourced from Tool Computer Use

The insight in this paper is methodological: function-calling for reasoning tasks is a domain where DPO outperforms SFT for small models, because the failure modes are more about preferring the right format and call sequence than about generating any plausible text. The proposed framework uses an agent that, given a problem and a callable function set, queries a large LLM by injecting function descriptions and examples and managing calls in a step-by-step reasoning chain. The byproduct is a dataset of correct AND incorrect chat completions — preference pairs ready for DPO.

Why DPO rather than SFT or PPO. SFT teaches the model to imitate good examples but provides no signal about what to avoid — and rigid output formats (precise variable names, JSON, argument values) punish near-misses harshly, so explicit negative examples matter. PPO would work but requires extensive human feedback to train a reward model, making it resource-intensive. DPO removes the reward-model step by incorporating preferences directly into the training objective, with demonstrated stability advantages over PPO.

The structural move is that a large LLM does double duty: it generates the candidate reasoning chains AND its successes/failures provide the preference labels for the small model's training. This is a teacher-distillation pattern but with both polarities — the small model learns what the large model gets right and what it gets wrong, not just to imitate the large model's right answers. The pattern fits the broader case for Can small language models handle most agent tasks?: function-calling is exactly the kind of repetitive, scoped, format-rigid work where a fine-tuned small model can replace a large general-purpose one.

The practical implication: when output format is rigid and small-model deployment is the goal, the question is not "can SFT close the gap" but "what's the cheapest source of preference signal." Self-generated preference pairs from a strong teacher are essentially free relative to human feedback.

Inquiring lines that read this note 113

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

How can AI alignment serve diverse human preferences at scale? Can alternative training methods improve on supervised fine-tuning for language models? What determines success in training models on multiple tasks? When does architectural design matter more than raw model capacity? Can prompting inject entirely new knowledge into language models? What structural factors drive popularity bias in recommendation systems? How should models express uncertainty rather than forced confident answers? Why does verification consistently lag behind AI generation? Does alignment training create blind spots in detecting genuine safety threats? Do language models learn genuine linguistic structure or just surface patterns? Does model scaling alone produce compositional generalization without symbolic mechanisms? Can inference-time compute substitute for scaling up model parameters? How effectively do deterministic tools improve language model reasoning on formal tasks? How does example difficulty affect learning efficiency in language models? How do training data properties shape reasoning capability development? Why does supervised fine-tuning improve accuracy while degrading reasoning quality? How can identical external performance mask different internal representations? Why do self-improving systems struggle without clear external performance metrics? What capability tradeoffs emerge when scaling model reasoning abilities? What are the consequences of models training on synthetic data? How do knowledge injection methods compare across cost and effectiveness? Does decoupling planning from execution improve multi-step reasoning accuracy? Why do reasoning models fail at systematic problem-solving and search? Why does training format shape reasoning strategy more than domain content? How do neural networks separate factual knowledge from reasoning abilities? What critical LLM failures do standard benchmarks hide? How do training priors constrain what context information can override? Does fine-tuning modify underlying model capabilities or only behavioral outputs? Why do continual learning scenarios trigger catastrophic forgetting and interference? What pretraining choices and baseline capability constrain reinforcement learning gains? Can model routing outperform monolithic scaling as an efficiency strategy? Do language models understand semantics or rely on pattern matching? What makes weaker teacher models effective for stronger student training? How should inference compute be adaptively allocated based on prompt difficulty? Why do semantic similarity and task relevance diverge in vector embeddings? Why does finetuning cause catastrophic forgetting of model capabilities? When do multi-agent approaches outperform single model extended thinking? How can AI agents autonomously learn and transfer skills across tasks? Can next-token prediction alone produce genuine language understanding? Should GUI agents use structured representations instead of raw pixels? Do harness improvements transfer across model scales or memorize shortcuts? Why do benchmark improvements fail to reflect actual reasoning quality? How do self-generated feedback mechanisms enable effective model learning? What role does compression play in language model capability and generalization?

Related concepts in this collection 5

This note in its neighbourhood — explore the map, then jump to a related concept in the list below.

Concept map
14 direct connections · 145 in 2-hop network ·dense cluster Open in graph ↗
Can small models match large models on function … Can small language models handle most agent tasks? Can breaking function calling into subtasks improv… Where do traditional function calling systems actu… Does teacher-refined data always improve student m… Why do alignment methods work if they model human …

Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph

your link semantically near linked from elsewhere

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Search by related questions 4

Suggested questions this note speaks to — click to search the collection, or type your own.

Why does DPO outperform SFT specifically for function calling tasks?How do self-generated preference pairs from a strong teacher compare to human feedback?Can granular sub-task training for function calling improve both open and proprietary models?What role does rigid output format play in function calling failure modes?
Original note title

DPO-trained small models can match large models on function-calling reasoning chains — preference data from a teacher beats SFT for the rigid output format