When does routing actually matter for multi-model systems?

Routers for multi-model systems are almost always judged on task accuracy and inference cost. "When is Routing Meaningful?" argues those metrics miss two structural properties that determine whether routing does anything at all. First, the society of actors must be behaviorally differentiated — if all actors respond identically, routing is vacuous, because there is nothing to choose between. Second, the policy must be stable — semantically-equivalent queries that differ only in spelling or syntax should reach the same actor, or the router is not specializing, just reacting to surface form.

The sharp point is that high accuracy is compatible with violating both. A router can score well over a redundant society (every model handles everything, so any assignment "works") or assign paraphrases inconsistently (accuracy survives because each model is competent), yet in both cases it has failed as a coordination mechanism — it is not distributing work in a way that reflects real specialization. This reframes a router as governance over a society, not a predictor of performance, and it distinguishes the diagnosis from Can routers select the right model before generation happens?: that note separates routing from reward modeling on the input side; this one shows even a good complexity-predictor can be meaningless if the society is redundant or the policy unstable.

Empirically the two properties trade off against performance in opposite ways. On EmbedLLM and RouterBench, KNN routers hit high accuracy on specialist societies but collapse under paraphrase perturbation, whereas prompted routers are less accurate but far more stable — so accuracy and robustness are in tension, and picking a router is picking a point on that frontier, not maximizing a single number.