Training an AI usually means copying a teacher first, then chasing a reward — but can both happen in a single step?

This question is really asking whether the classic two-step recipe — first copy a teacher (distillation/SFT), then push against a reward (RL) — can collapse into a single loop where both signals shape the same update. The corpus suggests the boundary between the two is thinner than the pipeline convention implies. The clearest bridge is on-policy distillation: instead of imitating a distant, fixed teacher, Can proximity between teacher and student fix distillation instability? constructs a *close* teacher and bounds the update inside a trust region — the same machinery RL uses to keep policy steps from blowing up. Once distillation borrows RL's trust-region discipline, it stops being a separate phase and starts behaving like one term in a policy update.

Coming from the other direction, several notes show reward optimization quietly absorbing the job distillation used to do. Can reinforcement learning embed domain knowledge more effectively than supervised fine-tuning? rewards both answer accuracy *and* explanation quality while cycling between augmented and unaugmented generation — so knowledge that SFT would have transferred by imitation gets internalized through the reward loop instead, and does it better because it optimizes reasoning rather than token-level copying. Can recommendation metrics train language models directly? goes further: training directly on rule-based metrics as RL rewards *eliminates* the SFT-distillation-from-a-proprietary-teacher step entirely. The lesson is that when a good reward exists, the distillation stage can be dissolved into it.

The deepest version is the model becoming its own teacher inside the loop. Can models learn to evaluate their own work during training? trains the model to compute its own reward in the unused sequence space after its output — self-evaluation and generation optimized together, with no external reward model at inference. That's distillation and reward optimization not just co-located but merged into a single self-referential objective.

If you do keep both signals as explicit, separate reward terms in one loop, the corpus is unusually concrete about how to stop them from fighting. Does binary reward training hurt model calibration? shows two objectives (accuracy + a calibration term) can be *jointly* optimized without trade-off when the second term is a proper scoring rule, and How should multiple reward objectives be weighted during training? gives the balancing recipe — weight each objective by its within-group reward variance so the high-signal term dominates automatically. Treat the distillation loss as just another objective and these tools apply directly.

The catch worth carrying away: fusing the two changes *what* gets transferred, not just efficiency. Does richer teacher context hurt student generalization? finds that teachers conditioned on answers and verifier output produce confident, concise traces students inherit — great in-domain, but they suppress the uncertainty-expression that out-of-distribution problems need. So a single loop that rewards a teacher's confident style can bake overconfidence straight into the policy. The unification is real and increasingly the default — but the reward you fold distillation into decides whether you're transferring competence or just transferring a teacher's blind spots.