Can proximity between teacher and student fix distillation instability?

On-policy distillation (OPD) has become the default LLM post-training paradigm because it occupies a sweet spot: it is mathematically equivalent to RL where the immediate reward is the log-probability ratio between teacher and student policies, so its on-policy nature avoids the catastrophic forgetting of SFT while its dense rewards escape the sample inefficiency and instability of RLVR (which gives only a sparse verifiable signal at the end). But standard OPD is optimization-fragile in practice, and TOP-D locates the bottleneck precisely: the capacity gap between a strong target teacher and a weaker student produces high-variance, unstable gradients.

The fix is to stop distilling directly from the distant target teacher and instead dynamically construct a proximal teacher — a teacher close to the current student — and iterate within a trust region. Theoretically this inherently controls gradient variance and, with safe off-policy data reuse inside the trust-region iterations, yields a formal global-convergence result and a monotonic-improvement bound. The framing is deliberately the classic RL move (TRPO's trust region) transplanted onto distillation: bound how far each update moves the policy, and stability follows for free — TOP-D adds zero computational overhead, decisively outperforming standard OPD and competitive RLVR baselines on mathematical reasoning across scales.

This connects to a growing recognition that the teacher/student gap is the load-bearing variable in distillation, not raw teacher strength. Since Does richer teacher context hurt student generalization?, the intuition that a maximally strong or maximally informed teacher is best keeps failing; proximity, not power, governs whether the transferred signal is learnable.