Picking which AI model handles each task sounds cheap — but does building that smart router cost more than it saves?

This explores whether a routing layer costs enough to build, run and maintain that it eats the savings from using cheaper models. One thing up front: the corpus has no study that adds up the engineering cost of a router and sets it against the money it saves. What it does have is evidence on each side of that ledger. That evidence suggests a router's expense comes less from building it and more from hidden costs: models that behave almost alike, unstable routing decisions, and new security exposure.

The headline savings are large. Routers that pick a model before any text is generated, by estimating how hard a query is, cut costs by 40–50% while adding very little delay, because only one model ever runs Can routers select the right model before generation happens?. Routing across groups of similar queries can match a frontier model at 27% lower cost, or beat it outright Can routing beat building one better model?. Real deployments go further. Banks test a task on an expensive model, then move the approved work to a cheap one. Santander reports cutting loan-approval costs from €5 to €0.03 per run Can banks cut AI costs by routing tasks to cheaper models?. Savings that size are hard to wipe out with engineering overhead. Also note that the bank pattern is a deliberate, human-run migration rather than a live router. Sometimes the cheapest routing layer is a process, not software.

The surprise for anyone worried about building costs is that sophisticated routers may not be worth it. A 400,000-query benchmark found that leading routing methods perform about the same, some commercial routers lose to a simple baseline, and lightweight clustering methods match routers that need neural-network training Do leading LLM routers actually outperform simpler alternatives?. The same benchmark found that routers fall short of a perfect router mainly because they miss which models could have answered correctly. Adding more models to the pool does not close that gap What's really holding back LLM routing performance?. So the real cost of over-building is engineering spent on complexity that buys nothing. Breaking routers into five standard parts helps teams compare designs before committing to one Can five components unify all LLM routing approaches?.

The less visible costs sit elsewhere. Routing only pays off if the models really behave differently. A router can score well on accuracy and still be meaningless if it sends two rephrasings of the same question to different models When does routing between models actually matter?. That kind of inconsistency is expensive to debug. The routing layer is also its own attack surface. Attackers can push requests toward weaker models, or make safety checks run against the wrong model identity Can attackers manipulate which model handles a request?. Securing that layer is a real cost that budgets often leave out. There is also a moving target: the cost of reaching a given capability level falls roughly 5–10x a year Does inference cost remain a persistent economic burden?. Today's price gap between expensive and cheap models may shrink faster than a complicated router pays for itself. And bigger models are not automatically better. Reasoning models show no consistent advantage on some numerical tasks Do reasoning models actually beat standard models on optimization?. That means more queries could go to cheap models than you might assume.

The point you might not expect: a router can earn back its cost in a second way. A deployed router records which queries went where and how each one turned out. That record can become labeled training data for fine-tuning or for distilling a cheaper model, which turns the router into a source of training data as well as a cost Can a routing harness generate its own training data automatically?. The better question is less whether the router pays for itself and more whether you are using what it learns.