Can distillation work on the student's own generated sequences?

Supervised knowledge distillation trains an auto-regressive student to imitate a fixed set of teacher output sequences, which creates a distribution mismatch: at training time the student sees teacher-authored sequences, but at inference it must continue from its own generations, and the two distributions diverge. Generalized Knowledge Distillation (GKD) fixes this by making distillation on-policy — the student generates sequences and the teacher provides feedback on those sequences. Because the training distribution is now the student's own output distribution, the exposure-bias gap closes: the student is corrected precisely on the mistakes it will actually make at inference. This is the same failure that RL solves for policy learning, imported into distillation.

Two consequences follow. First, GKD offers flexibility in the divergence used between student and teacher, which matters when the student lacks the expressivity to match the teacher's distribution exactly — forward-KL mode-covering can be swapped for objectives that degrade more gracefully under capacity limits. Second, learning from self-generated sequences with a reward signal means GKD composes with RL fine-tuning, so a single loop can distill teacher knowledge and optimize a sequence-level reward — a natural upgrade path for the RLHF phase. This connects to the risk surfaced in Does self-distillation harm mathematical reasoning performance?: on-policy distillation improves what distribution the student learns from, but it does not by itself protect the uncertainty tokens that distillation tends to strip, so the two concerns are orthogonal and both must be managed.