SYNTHESIS NOTE
Topics›Tool Computer Use›this note

Can small models match large models on function calling?

Explores whether small language models fine-tuned with the right training method can achieve comparable performance to large models on structured reasoning tasks requiring precise function calls, and what training approach makes this possible.

Synthesis note · 2026-05-03 · sourced from Tool Computer Use

The insight in this paper is methodological: function-calling for reasoning tasks is a domain where DPO outperforms SFT for small models, because the failure modes are more about preferring the right format and call sequence than about generating any plausible text. The proposed framework uses an agent that, given a problem and a callable function set, queries a large LLM by injecting function descriptions and examples and managing calls in a step-by-step reasoning chain. The byproduct is a dataset of correct AND incorrect chat completions — preference pairs ready for DPO.

Why DPO rather than SFT or PPO. SFT teaches the model to imitate good examples but provides no signal about what to avoid — and rigid output formats (precise variable names, JSON, argument values) punish near-misses harshly, so explicit negative examples matter. PPO would work but requires extensive human feedback to train a reward model, making it resource-intensive. DPO removes the reward-model step by incorporating preferences directly into the training objective, with demonstrated stability advantages over PPO.

The structural move is that a large LLM does double duty: it generates the candidate reasoning chains AND its successes/failures provide the preference labels for the small model's training. This is a teacher-distillation pattern but with both polarities — the small model learns what the large model gets right and what it gets wrong, not just to imitate the large model's right answers. The pattern fits the broader case for Can small language models handle most agent tasks?: function-calling is exactly the kind of repetitive, scoped, format-rigid work where a fine-tuned small model can replace a large general-purpose one.

The practical implication: when output format is rigid and small-model deployment is the goal, the question is not "can SFT close the gap" but "what's the cheapest source of preference signal." Self-generated preference pairs from a strong teacher are essentially free relative to human feedback.

Inquiring lines that read this note 134

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

Is model misalignment inherent to base models or induced by alignment procedures? How faithfully does iterative DPO approximate online RL? How do different post-training structures affect learning outcomes? How does scaling test-time compute compare to increasing model size? Does prompting inject new knowledge into models or activate existing knowledge? How does model scale affect fine-tuning and downstream task performance? Can structural constraints replace embedding depth in collaborative filtering? Does chain-of-thought reasoning faithfully reflect actual model reasoning processes? How does verifier integration affect the generation-verification gap? What reasoning limitations and biases do language models systematically exhibit? Does training data format shape reasoning more than domain content? Can scaling alone produce compositional generalization in neural networks? What makes reasoning models struggle with ill-posed and unfamiliar problems? What training data structures and signals produce robust learning across distributions? How do pretraining data composition and task diversity shape reasoning? Do language models truly understand their encoded knowledge or merely retrieve patterns? Does reasoning fine-tuning trade off other critical model abilities? What causes foundation models to develop task heuristics instead of understanding? How do benchmark scores relate to real-world task completion? Why does separating planning from execution enhance reasoning performance? Does parallel reasoning outperform sequential approaches in reasoning efficiency? Does reasoning architecture matter more than model scale? How can knowledge graph structure improve reasoning and retrieval? Do language models learn underlying linguistic structure or merely surface patterns? What are the trade-offs between memory capacity and continuous task learning? Does reinforcement learning create new reasoning capabilities or refine existing ones? How can parameter sparsity improve interpretability while maintaining model capability? When does routing outperform scaling as a performance solution? How do mechanistic features and representations drive model behavior? What determines whether visible reasoning traces improve task performance? What fundamental limitations prevent embeddings from solving task-relevant retrieval? Do standard benchmarks reveal AI capability in real-world contexts? What systematic failures occur in human preference-based model evaluation? How does explicit structure improve LLM reasoning on complex tasks? How do different editing strategies affect output quality and agent safety? Should GUI agents use structured UI perception instead of raw pixels? Does architectural recurrence enable computational capabilities that fixed-depth models cannot achieve? How do standardized artifacts improve coordination in multi-agent systems? When do multi-agent systems outperform single agents? Why do multiple language models independently generate similar outputs? How should organizational structure guide multi-agent system design?

Related concepts in this collection 5

This note in its neighbourhood — explore the map, then jump to a related concept in the list below.

Concept map
14 direct connections · 159 in 2-hop network ·dense cluster Open in graph ↗
Can small models match large models on function … Can small language models handle most agent tasks? Can breaking function calling into subtasks improv… Where do traditional function calling systems actu… Does teacher-refined data always improve student m… Why do alignment methods work if they model human …

Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph

your link semantically near linked from elsewhere

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Search by related questions 4

Suggested questions this note speaks to — click to search the collection, or type your own.

Why does DPO outperform SFT specifically for function calling tasks?How do self-generated preference pairs from a strong teacher compare to human feedback?Can granular sub-task training for function calling improve both open and proprietary models?What role does rigid output format play in function calling failure modes?
Original note title

DPO-trained small models can match large models on function-calling reasoning chains — preference data from a teacher beats SFT for the rigid output format