When is Routing Meaningful? Diversity and Robustness in Language Model Societies

Routing policies for multi-model systems are evaluated almost exclusively on task accuracy and inference cost. We argue that two properties, orthogonal to performance, determine whether routing is meaningful. First, the society of actors must be behaviourally differentiated: if all actors respond identically, routing is vacuous. Second, the routing policy must be stable: surface-form variants of a query should be assigned to the same actor. High task accuracy is compatible with violating both properties, since a router can operate over a redundant society or assign queries inconsistently, preventing specialisation regardless of performance. We adapt Hierarchic Social Entropy (HSE) to language-model societies and introduce a perturbation-based robustness metric to diagnose these failure modes. Applied to EmbedLLM and RouterBench, we find that HSE exhibits strong diminishing returns, suggesting that a curated subset of fewer than ten agents recovers most available diversity in a large pool — a practical coreset heuristic for society design.

Introduction. Routing is becoming a central mechanism for coordinating systems composed of multiple language models, tools, or agents (Wu et al., 2024; Hong et al., 2024; Li et al., 2023; Yue et al., 2025). Existing work on LLM routing largely evaluates routers by downstream utility: whether they improve task accuracy, reduce cost, or select the best-performing model for a query (Chen et al., 2024; Ong et al., 2025; Hu et al., 2024; Jiang et al., 2023). A router is not merely a predictor of task performance; it is also the coordination mechanism that determines how work is distributed across a society of actors. Its usefulness, therefore, depends on two structural properties. First, the society must contain behaviourally differentiated actors: if all actors respond in the same way, routing is vacuous (Balch, 2000; Bettini et al., 2025). Second, the routing policy should be robust to superficial variation: semantically-equivalent variants should not be sent to different actors merely because of changes in spelling, syntax, or wording (Sclar et al., 2024).

Discussion / Conclusion. We have argued that routing in language-model societies should be evaluated along two structural dimensions beyond task accuracy: behavioural diversity and routing stability under surface-form query variants. For diversity, we adapted Hierarchic Social Entropy to language-model societies, using cosine similarity over behavioural vectors as the inter-actor distance. For robustness, we introduced a perturbation-based metric measuring the fraction of surface-form variants assigned to the same actor. Applied to EmbedLLM and RouterBench, our analysis finds that HSE exhibits strong diminishing returns: fewer than ten agents capture most available diversity, providing a practical heuristic for society design. In addition, robustness and accuracy trade off sharply: KNN routers achieve high accuracy on specialist societies but collapse under perturbation, whereas prompted routers are less accurate but more stable across perturbation types and society configurations.