Cheap, small AI models can do most of an agent team's routine work, but where do they run out of steam?
Can small language models handle diverse tasks in heterogeneous multi-agent systems?
This explores whether small models can be the workhorses in agent teams built from mixed model sizes and types, and where that setup runs into limits.
This explores whether small models can be the workhorses in agent teams built from mixed model sizes and types, and where that setup runs into limits. For most of the work, the corpus says yes. Can small language models handle most agent tasks? argues that most of what agents do is repetitive, well-defined language work, and small models handle it at 10–30× lower cost. Its proposed design is small models by default, with a large model called in only when a step needs it. The catch is in the word 'diverse'. The evidence supports small models for diverse *well-defined* subtasks, not for open-ended ones.
Small models get there through training, not out of the box. Can small models match large models on function calling? shows that training on a large teacher's correct and incorrect function calls beats plain imitation, because the wrong examples target the rigid output-format failures small models are prone to. Can small agents learn from larger agents' memories? finds that copying a large agent's experience into a small one barely helps. Restructuring it into workflow, subtask and function-level memory lifted 4B–8B students by up to 27.2 points, sometimes close to the teacher, with no retraining. At the very small end, Does depth matter more than width for tiny language models? shows that deep, thin designs beat wide ones under a billion parameters. Can tiny recursive networks outperform massive language models? is a 7M-parameter puzzle solver, not a general language model. It still reached 45% on ARC-AGI-1, ahead of much larger LLMs, which suggests a narrow specialist can beat a big generalist on its own turf.
A mixed team only helps if its members actually differ, and the corpus is split on that. Do different AI models actually produce diverse outputs? tested 70+ models on open-ended questions and found an 'Artificial Hivemind': different models give strikingly similar answers because they share training data and alignment methods. Picking different models does not guarantee different perspectives. In structured settings, though, Do large language models use one reasoning style or many? finds real differences. Across 22 models, o1 leaned on minimax reasoning, DeepSeek-R1 on trust, and o3-mini on anticipating others' beliefs. The practical lesson is to build in variety through roles and task types, not by hoping different models will disagree usefully.
Where small models fall short, big models often do too. Do larger language models solve constrained optimization better? finds that LLMs stall at roughly 55–60% constraint satisfaction on optimization tasks regardless of size, so on those tasks handing the job to a bigger model is no fix. The other risk is the team itself. What limits autonomous capability in large language models? describes failure modes of multi-agent deliberation, such as Degeneration-of-Thought and Silent Agreement, that come from how agents interact, not from how big they are.
The corpus has no direct test of a small-model-heavy heterogeneous team on a broad task mix. The answer above is pieced together from cost analysis, training results and diversity findings. It suggests small models can carry the bulk of the work if they are trained for their roles, and that the open risks are shared blind spots and weak team coordination.
Sources 9 notes
SLMs handle the repetitive, well-defined language tasks that constitute most agent work at 10–30× lower cost than LLMs, making heterogeneous architectures (SLMs by default, LLMs selective) the economically rational design pattern.
Small models fine-tuned via DPO on correct and incorrect function-calling examples from a large teacher model achieve high accuracy on logical and mathematical tasks. DPO's explicit negative examples directly target the rigid output format failures where SFT alone underperforms.
Naive memory transfer from large to small agents yields only marginal gains, but restructuring the teacher's trajectories into workflow, subtask, and function memory—without retraining—closes the gap significantly. Students of 4B to 8B parameters achieved up to 27.2 percentage point gains and sometimes approached teacher-level performance.
MobileLLM shows deep-and-thin architectures yield 2.7–4.3% accuracy gains over balanced designs at 125M–350M scale by composing abstract concepts through layers rather than spreading parameters across width.
A 7M-parameter two-layer network recursing on its latent reasoning state reached 45% on ARC-AGI-1, beating larger LLMs with 0.01% of their parameters. The gains come from recursion itself, not scale or hierarchical architecture.
Show all 9 sources
INFINITY-CHAT analyzed 70+ models across 26K open-ended queries and found an "Artificial Hivemind" effect: models independently generate strikingly similar or identical responses due to overlapping training data and alignment procedures, undermining the diversity benefits of model ensembles.
Analysis of 22 LLMs across behavioral game theory reveals three dominant profiles: GPT-o1 uses minimax reasoning, DeepSeek-R1 uses trust-based reasoning, and GPT-o3-mini uses belief-anticipation. Performance correlates with game structure, not raw reasoning depth.
Across constrained-optimization tasks, LLMs converge to ~55–60% constraint satisfaction independent of architecture, parameter count, or training regime. Reasoning models do not systematically outperform standard models, suggesting a fundamental ceiling rather than a scaling gap.
Multi-agent deliberation produces specific failure modes (Degeneration-of-Thought, Silent Agreement), alignment at scale includes problematic self-valuation, and self-improvement is formally bounded by the generation-verification gap. Measurement error and conditional compliance hide the true capability ceiling.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- ZebraLogic: On the Scaling Limits of LLMs for Logical Reasoning
- VibeThinker-3B: Exploring the Frontier of Verifiable Reasoning in Small Language Models
- Small Language Models are the Future of Agentic AI
- MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases
- Scaling Behavior of Single LLM-Driven Multi-Agent Systems
- AI4AI at Test-Time: Strong-to-Weak Capability Transfer via Harnesses
- Hierarchical Reasoning Model
- Scaling Laws Meet Model Architecture: Toward Inference-Efficient LLMs