What makes routing a better investment than training larger models?
This explores why directing each query to the model best suited for it (routing) often pays off more than simply building one bigger model — and the corpus suggests selection is a stronger lever than scale.
This explores why routing — sending each query to the model best suited for it — can be a smarter bet than pouring resources into a single larger model. The clearest evidence is direct: a routing system that sends queries to the best model per semantic cluster beats a frontier model by 7% on accuracy, or matches it at 27% lower cost, and earlier work showed ten small 7B models with good routing surpassing far larger frontier models Can routing beat building one better model?. The lesson isn't that small models are secretly great — it's that *choosing the right model for each query* extracts more value than making any one model bigger. Selection is the lever; scale is just one thing you can select for.
Part of why routing wins is that scaling has ceilings the corpus keeps running into. On genuine constrained-optimization tasks, models plateau at 55–60% regardless of parameter count, architecture, or training regime — a wall, not a gap a bigger model closes Do larger language models solve constrained optimization better?. And bigger isn't always better even when it works: pushing models with harder training can backfire, where overly difficult RLVR samples teach degenerate shortcuts that contaminate abilities the model already had Do overly hard RLVR samples actually harm model capabilities?. When more scale and more training hit diminishing or negative returns, spending your effort on *which* model answers becomes the higher-yield investment.
Routing is also cheap in a way training never is. It's a pre-generation decision — a router estimates how hard a query is and picks a model *before* any tokens are generated, cutting cost 40–50% with minimal latency because nothing has to be run twice or evaluated after the fact Can routers select the right model before generation happens?. Contrast that with the price of training: a larger model costs more on every single query forever, whether or not that query needed the horsepower. Routing lets you pay for capability only when the query demands it.
The deeper point the corpus makes is that capability is mostly determined before inference — so where you intervene matters. Non-reasoning models can't catch up to reasoning models no matter how much inference compute you give them, because the training regime, not raw size, installs the protocol that makes extra tokens productive Can non-reasoning models catch up with more compute?. And small models can be cheaply lifted to match large ones on specific tasks — DPO training on a teacher's good and bad examples gets small models to large-model accuracy on function calling Can small models match large models on function calling?. Put those together and a portfolio of cheaply-specialized models plus a smart router starts to look strictly better than one monolith.
What's surprising is how far this 'selection over scale' idea generalizes beyond model picking. You can route *within* a model's behavior too: chain-of-thought verbosity turns out to be a single steerable direction in activation space, so you can cut reasoning length 67% with no retraining at all Can we steer reasoning toward brevity without retraining?. And the same logic is becoming infrastructure for multi-agent systems, where versioned capability vectors let you discover and route to the right agent automatically instead of hand-wiring connections Can semantic capability vectors replace manual agent routing?. The throughline: matching work to the right resource — query to model, query to brevity, task to agent — repeatedly beats making the resource bigger.
Sources 8 notes
Avengers-Pro achieves 7% higher accuracy than GPT-5-medium by routing queries to optimal models per semantic cluster, or matches its performance at 27% lower cost. Ten 7B models with routing previously surpassed GPT-4.1 and 4.5, suggesting selection is a stronger lever than scaling.
Across constrained-optimization tasks, LLMs converge to ~55–60% constraint satisfaction independent of architecture, parameter count, or training regime. Reasoning models do not systematically outperform standard models, suggesting a fundamental ceiling rather than a scaling gap.
Training on nearly-impossible problems causes models to learn degenerate shortcuts rather than genuine reasoning, and these shortcuts contaminate pre-existing capabilities. Group-relative normalization treats rare accidental successes as high-advantage trajectories, reinforcing answer repetition and computation-skipping instead of sound reasoning patterns.
RouteLLM and Hybrid-LLM both achieve 40-50% cost reduction by routing to a single model based on query difficulty prediction, not response evaluation. Single-model routing minimizes latency compared to ensemble or cascade alternatives.
Reasoning models persistently outperform non-reasoning models regardless of inference budget because training instills a reasoning protocol that makes additional tokens productive. The gap is fundamentally about deployment mechanisms and training structure, not raw capability.
Small models fine-tuned via DPO on correct and incorrect function-calling examples from a large teacher model achieve high accuracy on logical and mathematical tasks. DPO's explicit negative examples directly target the rigid output format failures where SFT alone underperforms.
Activation-Steered Compression extracts a single vector from 50 paired examples to reduce chain-of-thought length by 67% while maintaining accuracy and achieving 2.73x speedup. The method is training-free and generalizes across model sizes and domains.
Versioned Capability Vectors embedded in HNSW indices couple semantic matching with policy and budget constraints, making capability discovery a first-class operation that scales sub-linearly as agent heterogeneity increases.