SYNTHESIS NOTE
Topics›Routers›this note

Can routing beat building one better model?

Does directing queries to specialized models via semantic clustering outperform investing in a single frontier model? This challenges whether model improvement or model selection drives performance gains.

Synthesis note · 2026-02-23 · sourced from Routers

Avengers-Pro demonstrates that routing queries to different models based on semantic clustering can exceed the performance of any individual model in the pool — including frontier models. The mechanism: embed incoming queries, cluster by semantic similarity, evaluate per-cluster model performance-efficiency scores, and route each query to the highest-scoring model for its cluster.

Three results establish the claim:

The earlier Avengers work made an even more striking claim: ten models of ~7B parameters each, with routing, surpassed GPT-4.1 and 4.5 across 15 datasets. This suggests the performance gain from optimal model selection can be comparable to the gap between model generations.

The architecture is lightweight: three operations at inference time (embedding, nearest-cluster lookup, score aggregation). The heavy work — fitting the clustering model and estimating per-cluster performance statistics — happens offline on a calibration set (70% for fitting, 30% for evaluation). This makes the approach deployable as a thin routing layer atop any model API ecosystem.

Since Can we allocate inference compute based on prompt difficulty?, Avengers-Pro adds a complementary optimization axis. Compute-optimal scaling asks "how much inference budget per query?" Routing asks "which model per query?" These are independent — a routing layer could be composed with per-query compute allocation for a two-dimensional Pareto optimization. Since Can inference compute replace scaling up model size?, routing extends this: you don't need a bigger model OR more compute — you need the right model for this specific query type.

The implication challenges the frontier model race: rather than building one model that dominates on everything, assembling a diverse pool of specialized-ish models with good routing may be both cheaper and more effective. This aligns with the heterogeneous architecture thesis in Can small language models handle most agent tasks? — routing makes the heterogeneous approach practical.

Inquiring lines that read this note 93

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

How should conversational recommenders decide when to ask versus recommend? When does routing outperform scaling as a performance solution? How can recommendation systems balance individual and population-level goals? Does architectural recurrence enable computational capabilities that fixed-depth models cannot achieve? What systematic failures occur in human preference-based model evaluation? What is the actual bottleneck preventing effective AI research automation? How do diffusion language models compare to autoregressive architectures? How does model scale affect fine-tuning and downstream task performance? When do multi-agent systems outperform single agents? When should retrieval be triggered to augment language model reasoning? Does parallel reasoning outperform sequential approaches in reasoning efficiency? How should organizational structure guide multi-agent system design? How should context management strategies balance efficiency with agent reliability? How do frontier models develop behaviors that escape detection? Why does self-revision increase model confidence despite degraded accuracy? How should test-time compute be allocated adaptively across task difficulties? Can synthetic data preserve necessary diversity to prevent model collapse? What fundamentally drives AI inference pricing and cost decline patterns? Does semantic diversity improve reasoning quality in reinforcement learning? What improves LLM forecasting accuracy beyond base model capability? How can language models effectively manage long-context memory without quality degradation? Why does separating planning from execution enhance reasoning performance? Can scaling alone produce compositional generalization in neural networks? Do benchmark score improvements actually reflect genuine AI capability gains? Does evolutionary optimization at test time improve model performance effectively? What determines when personalization helps versus harms model behavior? Can harness optimization rival model scaling in effectiveness? What organizational practices enable successful AI tool adoption?

Related concepts in this collection 5

This note in its neighbourhood — explore the map, then jump to a related concept in the list below.

Concept map
15 direct connections · 133 in 2-hop network ·medium cluster Open in graph ↗
Can routing beat building one better model? Can we allocate inference compute based on prompt … Can inference compute replace scaling up model siz… Can small language models handle most agent tasks? Can routers select the right model before generati… Can attackers manipulate which model handles a req…

Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph

your link semantically near linked from elsewhere

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Search by related questions 4

Suggested questions this note speaks to — click to search the collection, or type your own.

How does semantic clustering help decide which model handles each query?Can compute allocation and model routing be combined for better results?Why might diverse smaller models with routing beat one giant model?What makes routing a better investment than training larger models?
Original note title

test-time model ensembling via embedding-cluster routing surpasses any individual frontier model — model selection is a stronger lever than model improvement