Can a model's own consensus replace ground truth labels?

The paper's opening complaint is that existing on-policy (self-)distillation "still rel[ies] heavily on external supervision" — ground-truth signals, environmental feedback, or guidance from larger models — and so "fall[s] short of genuine 'self'-distillation." Its answer, unsupervised on-policy self-distillation (U-OPSD), removes all three. With the OPSD recipe "otherwise untouched," the ground-truth solution is replaced by the model's own majority-vote consensus, and the label-free variant "outperforms its supervised counterpart on the five-benchmark average." The abstract goes further and says U-OPSD "matches or surpasses" GT-supervised methods such as OPSD and GRPO.

The mechanism has two gates. The model samples multiple rollouts and builds a pseudo solution by majority vote "under a self-consistency threshold," so a consensus exists only when the model agrees with itself often enough. The model's distribution is then conditioned on that pseudo solution, and distillation is applied "only to self-inconsistent rollouts." The agreeing rollout is the teacher's reference; the disagreeing completions are the training targets. That is how the paper describes correcting the model "precisely where it is confidently wrong": the signal comes from the gap between what the model usually says and what it sometimes says instead.

The discussion states its own scope condition. The advantage reproduces at both model sizes tested in non-thinking mode and is a tie in thinking mode, which the authors attribute to the base model already being strong enough that little headroom remains. Their conclusion is that in the regime of "a base model competent enough to vote well and fallible enough to have headroom," the binding constraint "is not access to gold solutions but the machinery for surfacing and correcting the model's own inconsistencies."

Against the neighbors, this pushes the self-teacher idea one step further. Can environment feedback replace scalar rewards in policy learning? still needs the environment to supply the feedback the policy conditions on; U-OPSD supplies the conditioning content from the model's own vote, and the abstract lists environmental feedback among the supervision that falls short of "genuine" self-distillation. In the terms of Can language models replace reward models with internal signals?, it looks like a further substitution: consensus stands in for the verifier or label while the distillation machinery stays put. It also inherits the on-policy benefit described in Can distillation work on the student's own generated sequences?, which the introduction cites as the reason to distill on the model's own generations.

The excerpt does not establish several things. It names no benchmarks, base models, sizes, effect sizes, threshold value, or rollout count, and says nothing about compute cost against supervised training. It does not say what happens when the majority answer is wrong, so how far the threshold protects against reinforcing a confident error is unreported here. It does not test what the label-free teacher does to reasoning length or out-of-distribution behavior, the trade-off flagged in Does richer teacher context hurt student generalization?. What follows at the strength the evidence allows is narrow: for models that already vote well and still have headroom, gold solutions are not the limiting input to on-policy self-distillation. For saturated or weak models the excerpt gives no support.