Should an AI learn only from problems where its own answers disagree, or from everything it writes?

This explores whether a model learns better when self-distillation trains only on the questions where its own attempts disagree, compared with training on everything it generates. The library has one note on the selective approach, Can a model's own consensus replace ground truth labels?, and it doesn't include a head-to-head test against distilling all generations. That note reports that unsupervised on-policy self-distillation matched or beat supervised methods on five benchmarks. It gets there by distilling only on self-inconsistent rollouts, using the model's majority-vote agreement as the teaching signal instead of ground-truth labels. Nothing in the library shows what happens if you distill on the consistent rollouts too, so anything beyond that is inference from neighboring notes.

The standard alternative is on-policy distillation over everything: the student generates sequences and a teacher scores all of them, as in Can distillation work on the student's own generated sequences?. That approach fixes the mismatch between training and real use, but it treats every generation as equally worth learning from. A neighboring finding suggests why that may be wasteful: Does on-policy distillation actually expand student capability? says this kind of distillation mostly steers a student toward correct paths it can already reach, and that signal quality and diversity matter more than teacher size. If the job is choosing between reasoning paths the model can already produce, the questions where its samples split are where the choice exists. Where every sample agrees, there is little left to steer.

The selective version also sidesteps two problems that appear in the library's other distillation notes. Does richer teacher context hurt student generalization? shows that a teacher given the answer produces confident, concise traces, and students inherit that confidence, which hurts them on out-of-distribution problems that call for caution. A model teaching itself through majority vote has no all-knowing teacher to copy, and the training concentrates on the spots where it was actually uncertain. Separately, Can proximity between teacher and student fix distillation instability? finds that distillation destabilizes when the teacher is far from the student. A teacher built from the student's own consensus is about as close as a teacher can be, though the library doesn't test that link.

The main risk for the selective approach is that agreement is not the same as being right. Does setting temperature to zero actually make LLM outputs reliable? makes a related point about repeated outputs: consistency can just mean the same draw came out again. A model that is consistently wrong on a question would be skipped entirely, and its errors would never be corrected. Majority vote also assumes the most common answer among inconsistent rollouts is the better one. The five-benchmark result suggests that assumption held there, but it isn't proof it holds everywhere.

A related idea shows up in agent training. Can tree structure alone convert outcome rewards into process supervision? gets step-level supervision by comparing sibling branches of the same rollout tree, so the signal again comes from samples that diverge. The pattern across these notes is that disagreement between a model's own samples is treated as the informative part. Whether that beats using every generation is the open question, and the library doesn't answer it.