An AI 'router' picks which model answers each question — but what should it learn from to choose well?

This explores what a router should learn from, and whether one signal holds up across benchmarks. The corpus has no note that compares supervision signals head-to-head across tasks, so it can't name a winner. It does show what the leading candidates are and where each one comes from.

The most direct candidate is difficulty prediction. RouteLLM and Hybrid-LLM both train a router to estimate how hard a query is before any model answers, then send it to a single model. Both cut cost by 40-50% this way, and the router never has to evaluate a response Can routers select the right model before generation happens?. The training label is therefore roughly 'would the cheap model have been good enough?', not 'which answer was best?'. This is the cheapest signal to use at serving time, because the router only reads the query.

A second candidate is outcome data collected from a deployed system. A working routing harness already records execution trajectories, estimates of how much capability each task demanded, and whether the result succeeded. Those records convert into labeled examples for fine-tuning and distillation, so the harness doubles as a difficulty labeler Can a routing harness generate its own training data automatically?. The signal comes from real usage across whatever tasks show up, not from a fixed benchmark. That makes it a promising answer to 'across tasks', though the corpus presents it as a mechanism, not a measured result. The LLMRouter framework helps with the comparison problem. It breaks routers into five components under one sequential-decision view, so single-turn, multi-turn and personalized routers can be compared fairly Can five components unify all LLM routing approaches?. Without that, a claim like 'this signal is best' may only reflect a different router design.

Two neighboring notes suggest less obvious signals, and both are extrapolations from outside routing. First, an RL note shows one statistic doing two jobs. Variance across a model's repeated attempts weights individual tokens, and it also filters out queries where the comparison carries no information, which trained 2-3× faster on tasks with no verifiable answer Can one statistical measure serve dual purposes in RL training?. The router analogue is that queries where every candidate model succeeds, or every one fails, teach a router nothing about which model to choose. The most informative training examples are the ones where models disagree. Second, Branch-Train-MiX learns routing at the token level, inside a merged model, using only the ordinary language-modeling loss, with no hand-labeled difficulty Can asynchronous expert training beat synchronized distributed LLM training?. That is a different kind of router and a different kind of supervision. It shows that 'router training' covers both query-level selection among whole models and token-level selection among experts.

In short, the corpus gives three signal sources: predicted difficulty, logged outcomes from deployment, and disagreement between models used as a filter. It offers a way to compare routers fairly, but no evidence yet on which signal transfers best across benchmarks. That gap is probably the most useful thing to know before trusting a claim that one signal is best.