Should an AI agent use one powerful (expensive) model for everything, or mix in cheaper, smaller models where it can?

This explores whether agent systems should mix cheap small models with expensive large ones instead of sending every step to a frontier model. The corpus mostly says yes. Its reasoning comes less from model rankings and more from what agent work actually consists of. Most of what an agent does is narrow and repetitive: parsing a tool output, formatting a call, routing a request, filling a template. Small language models handle these tasks well at roughly 10–30× lower cost, so the rational default is to let small models do most of the work and call a large model only when a step genuinely needs it Can small language models handle most agent tasks?.

The less obvious point is why so many teams build the opposite way. If a benchmark scores only accuracy, nothing pushes back against expensive designs, and leaderboard agents drift toward needless complexity and cost. When cost and accuracy are scored together, you can often find a much cheaper system that is just as accurate, especially once the cost is spread over many repeated runs Why do accuracy-only benchmarks hide the true cost of agents?. A related critique goes further. Agents that win benchmark contests often fail at long, real professional workflows, so a big model's leaderboard score is weak evidence that it is the right model for paid work Why do agent benchmarks not predict real economic value?.

There is also a cost that compounds over time. In multi-step work, every call adds cost and delay, and both pile up across the whole task. A 35B-parameter model trained specifically on execution and follow-through reaches the low-cost sweet spot of the cost-performance tradeoff while competing with much larger models. That suggests training for coordination can partly replace raw scale Does model efficiency matter more than peak capability for real work?. The scaffolding around a model matters too. Improving only the execution harness raises results for fixed models without changing their weights Can execution harnesses lift model performance without retuning weights?. More of the value then sits in the system, and less in which model you picked.

Here is the twist you might not expect: the biggest model isn't always the best fit even for the system's own machinery. When agents evolve their own harnesses, models at every tier write useful edits about equally well. The models that benefit most from those edits, though, are mid-tier. Weak models fail to use the harness, and the strongest ones are worse at following its instructions faithfully Do stronger models always evolve harnesses better?. So mixing models isn't only about saving money. Different tiers have different strengths, and a smart system places each one where it fits.

The open frontier is making that placement automatic. Meta-agents trained with reinforcement learning can design a different multi-agent setup for each query, balancing performance, complexity and efficiency, instead of reusing one fixed template Can AI systems design unique multi-agent workflows per individual query?. This fits a wider trend: recent progress in self-improving agents comes mostly from cheap, reversible changes to prompts, tools and memory, not from expensive retraining of weights Do self-improving agents really split into two distinct loops?. Choosing which model handles which step is exactly that kind of cheap, reversible change. The corpus has little hard data on how well routing between models works in real deployments, so the economic case is strong on principle but thinner on measured evidence.