Every time an AI routes a question to the right model, it leaves clues about difficulty. Can those clues sort training data automatically?
Can routing signals organize training data into a meaningful curriculum automatically?
This explores whether the by-products of routing (which queries go where, how hard they look, whether they succeed) can be used to sort training data into an easy-to-hard or skill-by-skill order without a person designing it.
This explores whether the by-products of routing, meaning which queries go where, how hard they look and whether they succeed, can sort training data into a curriculum without a person designing it. The corpus says the raw ingredients exist. It also shows that a routing-derived curriculum is a proposal, not yet a tested result. The clearest statement is that a deployed routing harness already records execution trajectories, estimates of how much capability each query demands, and outcomes. These can be turned into labeled training examples, so the harness becomes a serving component and a difficulty labeler at once Can a routing harness generate its own training data automatically?. Difficulty is the label a curriculum needs most, and here it comes free from a router doing its normal job.
Routers also produce a second kind of structure: grouping. Avengers-Pro clusters queries by meaning and sends each cluster to whichever model does best on it, which beats a single frontier model Can routing beat building one better model?. Those clusters amount to a topic map of the query stream. Each cluster's per-model win rates show where the models are weak. That would be a natural way to decide what to train on next. This is my extrapolation, not something the note claims.
The corpus has good examples of what an automatic curriculum looks like once you have a difficulty signal, even though none of them use a router. Reverse-curriculum RL starts the model near the finish of a reasoning chain and slides the starting point backward. Practice moves from easy endings to hard beginnings, and step-level failures show up using only outcome feedback Can curriculum learning approximate expensive process supervision?. DRO takes a different route. It measures how much a model's own attempts disagree, then throws out queries where they all agree, because there is nothing to learn from them. Training gets 2–3× faster Can one statistical measure serve dual purposes in RL training?. In both, the model's own pattern of successes and failures defines difficulty. A router's deployment record is the same kind of signal, collected on real traffic.
Branch-Train-MiX runs the other way. Humans split the data by domain first, experts train in parallel, and a router is learned afterward to combine them Can asynchronous expert training beat synchronized distributed LLM training?. Routing there follows the data organization and doesn't produce it. Making routing the organizer is the step the corpus leaves open.
Two other notes warn about what such a curriculum might end up sorting. RL training tends to amplify one dominant pretraining format and suppress the rest Does RL training collapse format diversity in pretrained models?. Instruction tuning appears to teach the shape of the output more than understanding of the task Does instruction tuning teach task understanding or output format?. An automatic curriculum built from surface features could therefore mostly reorder formats. Signals based on outcomes, like success rates and disagreement across attempts, are the safer basis for ordering by skill. The corpus has no direct experiment comparing a router-derived curriculum with a hand-built one.
Sources 7 notes
A deployed routing system records execution trajectories, capability demand estimates, and outcome data that can be converted into labeled training examples for fine-tuning and distillation, turning the harness into both a serving component and a difficulty labeler.
Avengers-Pro achieves 7% higher accuracy than GPT-5-medium by routing queries to optimal models per semantic cluster, or matches its performance at 27% lower cost. Ten 7B models with routing previously surpassed GPT-4.1 and 4.5, suggesting selection is a stronger lever than scaling.
R3 progressively slides the reasoning start state backward from near-completion, creating a curriculum that reveals step-level failure modes using only outcome feedback. This achieves process supervision granularity without expensive human step annotations.
DRO reuses a single self-supervised statistic at two aggregation levels: token-level weighting in dense rewards and query-level filtering to discard degenerate comparisons. This dual use achieves 2–3× faster training with better stability on unverifiable tasks.
Branch-Train-MiX trains domain experts in parallel without synchronization overhead, merges their feed-forward parameters as MoE experts, and learns token-level routing, achieving better accuracy-efficiency tradeoffs than synchronized training or routing-free merging.
Show all 7 sources
Controlled experiments show RL consistently amplifies one format distribution from pretraining within the first epoch while collapsing alternatives. The winning format depends on model scale, not necessarily performance, and is largely hidden when starting from proprietary pretrained models.
Models trained on semantically empty or deliberately incorrect instructions achieve comparable performance to those trained on full correct instructions, achieving 43% vs random baseline 42.6%. The semantic content of instructions appears largely irrelevant; what transfers is knowledge of the output space.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Echo Chamber: RL Post-training Amplifies Behaviors Learned in Pretraining
- Planted in Pretraining, Swayed by Finetuning: A Case Study on the Origins of Cognitive Biases in LLMs
- The Art of Scaling Reinforcement Learning Compute for LLMs
- Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing
- RouteLLM: Learning to Route LLMs with Preference Data
- LLMRouter: Unified Infrastructure for Developing, Evaluating, and Deploying LLM Routers
- A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?
- Do Models Really Learn to Follow Instructions? An Empirical Study of Instruction Tuning