When a system routes each question to the right AI model, which model actually does the teaching if you distill it?
Who serves as the teacher model in the routing-guided distillation process?
This explores who plays the teacher in a distillation pipeline guided by a router (a system that picks which model handles each query). The corpus has no note that names a specific teacher for one, so this answer covers what the nearest notes say and where the gap is.
This explores who plays the teacher in a distillation pipeline guided by a router. The corpus never names one. The closest note describes a deployed routing harness that records execution trajectories, estimates of how much capability each query demands, and outcome data. That record can be turned into labeled training examples for fine-tuning and distillation, so the harness becomes both a serving component and a difficulty labeler (Can a routing harness generate its own training data automatically?). In that picture the router labels which queries are hard, and it does not seem to be the teacher. The note doesn't say which model supplies the teaching signal. A natural guess is the stronger models the router sent hard queries to, but that is an inference, not something the corpus states.
Routing itself doesn't require a teacher. Routers like RouteLLM and Hybrid-LLM choose a model before any text is generated, based on predicted query difficulty (Can routers select the right model before generation happens?). One framework breaks every router into five components that can be compared directly (Can five components unify all LLM routing approaches?). Nothing in either note connects routing to distillation. The link only shows up when a harness's logs are reused as training data.
The distillation notes suggest the teacher may matter less than the question assumes. On-policy distillation gets much of its gain from pushing probability away from unlikely tokens, and a fixed penalty with no teacher reproduces that benefit (Does on-policy distillation need a teacher at all?). A smaller teacher with high-fidelity guidance can beat a larger one without it, so signal quality and diversity count for more than teacher scale (Does on-policy distillation actually expand student capability?). If a routing pipeline does distill, the biggest model it routes to is not automatically the best teacher.
The corpus also shows several other ways to fill the teacher role. A model can teach itself: its own majority-vote consensus, used only on rollouts where it disagrees with itself, matched or beat supervised methods on five benchmarks (Can a model's own consensus replace ground truth labels?). A teacher can be built to sit close to the student, inside a trust region, which stabilizes training compared with distilling from a distant target (Can proximity between teacher and student fix distillation instability?). A teacher can be given the correct answer and verifier output. That yields confident, short traces that students inherit, at the cost of the uncertainty expression they need on out-of-distribution problems (Does richer teacher context hurt student generalization?). In the standard on-policy setup the teacher scores sequences the student generated itself (Can distillation work on the student's own generated sequences?).
So the honest answer is that the corpus doesn't say. If you meet a specific routing-guided distillation paper, check who the teacher is: a stronger routed-to model, the student's own consensus, or a constructed proximal teacher. The corpus suggests that choice shapes the result more than the routing does.
Sources 9 notes
A deployed routing system records execution trajectories, capability demand estimates, and outcome data that can be converted into labeled training examples for fine-tuning and distillation, turning the harness into both a serving component and a difficulty labeler.
RouteLLM and Hybrid-LLM both achieve 40-50% cost reduction by routing to a single model based on query difficulty prediction, not response evaluation. Single-model routing minimizes latency compared to ensemble or cascade alternatives.
The LLMRouter framework casts routing as a sequential decision process with five component types, enabling fair comparison of diverse routers and unifying single-turn, multi-turn, and personalized routing as instances of a common design space.
Research shows that on-policy distillation's gains come largely from pushing probability away from low-likelihood tokens, a benefit that a simple fixed penalty reproduces without any teacher. Teacher supervision contributes substantial noise that students ignore.
On-policy distillation steers students toward correct reasoning paths within their existing capability envelope rather than raising the ceiling. Signal quality and diversity matter far more than teacher scale; a smaller teacher with high-fidelity guidance outperforms larger teachers without it.
Show all 9 sources
Unsupervised on-policy self-distillation using the model's own majority-vote consensus matched or surpassed supervised methods on five benchmarks. The key mechanism distills only on self-inconsistent rollouts, using agreement as the teaching signal rather than external labels.
TOP-D constructs a close teacher instead of distilling from a distant target, bounded by a trust region. This controls gradient variance, guarantees monotonic improvement, and outperforms standard distillation with zero computational overhead.
Teachers conditioned on correct answers and verifier output produce confident, concise traces that students inherit. This style suppresses uncertainty expression, optimizing in-domain performance while degrading generalization to out-of-distribution problems that require epistemic caution.
Generalized Knowledge Distillation (GKD) fixes the train-inference mismatch by having students generate sequences that the teacher then scores, aligning the training distribution with the student's actual inference behavior. This on-policy approach also composes cleanly with RL fine-tuning.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- On-Policy Self-Distillation without Any Supervision
- Why Does Self-Distillation (Sometimes) Degrade the Reasoning Capability of LLMs?
- Does On-Policy Distillation Really Distill? From Noisy Teacher to Self-Improvement
- Self-distillation Enables Continual Learning
- Trust Region Policy Distillation
- AI4AI at Test-Time: Strong-to-Weak Capability Transfer via Harnesses
- Demystifying On-Policy Distillation: Roles, Pathologies, and Regulations
- Reinforcement Learning via Self-Distillation