What information is lost when majority labels discard minority interpretations?
This explores what gets thrown away when we collapse many human judgments into a single 'correct' label — and whether that discarded signal was noise or actually meaningful.
This explores what gets thrown away when we collapse many human judgments into one majority label. The corpus suggests the discarded material is often not error but information — and that treating disagreement as noise quietly corrupts everything trained on top of it.
The sharpest claim comes from Interpretation Modeling research: for socially embedded sentences, disagreement between annotators reflects valid differences in reader perspective, not annotation failure Why do readers interpret the same sentence so differently?. The spread of interpretations *is* the data. When you majority-vote it down to one answer, you don't resolve the ambiguity — you erase the fact that it existed, and with it the social position that produced each reading.
There's a second, subtler loss. Not every annotation is even measuring the same thing. One analysis decomposes annotation responses into three distinct signals — genuine preferences, non-attitudes, and constructed-on-the-spot preferences — distinguishable only by how consistent they are across conditions Do all annotation responses measure the same underlying thing?. A majority label flattens all three into one number, so a confident stable judgment and a coin-flip guess count equally. That contamination flows downstream into reward models and alignment.
The same erasure shows up in reasoning, not just labeling. Self-consistency voting picks the most common final answer and discards the intermediate reasoning of every losing chain — yet meta-reasoning *over* all the chains beats voting on both accuracy and interpretability, because the minority chains carried distributed information the winner didn't Does voting discard useful reasoning from losing chains?. The lost thing is the same shape as in annotation: the auditable trail of how different paths got somewhere.
What makes this genuinely tension-filled is that majority voting also *works*, sometimes spectacularly. Models trained on many biased experts converge toward a consensus that outperforms any single expert, because averaging denoises uncorrelated individual errors Can models trained on many imperfect experts outperform each one?, and models can even bootstrap self-improvement on unlabeled data using majority-vote rewards, since consensus answers tend to be correct Can models improve themselves using only majority voting?. So the real lesson isn't 'never use the majority.' It's that majority voting is the right tool when disagreement is *uncorrelated error* and the wrong tool when disagreement is *structured perspective* — and the corpus keeps finding cases where we've mistaken the second for the first.
Sources 5 notes
Interpretation Modeling research shows that disagreement on socially embedded sentences reflects valid differences in reader perspective, not annotation failure. Structured human disagreement in NLI benchmarks confirms that interpretation distributions carry meaningful information.
Behavioral science reveals that annotations contain genuine preferences, non-attitudes, and constructed preferences—distinguishable by consistency across measurement conditions. Treating them uniformly contaminates reward model training and downstream alignment.
Standard self-consistency voting selects the majority answer but discards intermediate reasoning from non-winning chains. Multi-chain reasoning instead meta-reasons over all chains simultaneously to extract distributed information, improving both task accuracy and producing coherent, auditable explanations.
Generative models trained on many diverse experts with different biases converge toward consensus behavior through cross-entropy optimization. Low-temperature sampling reveals this implicit majority vote, which outperforms any single expert by denoising uncorrelated individual errors on critical decision states.
Test-Time RL generates reward signals by majority voting across repeated samples, enabling policy improvement without ground-truth labels or trained reward models. This approach works surprisingly well because consensus answers tend to be correct, creating a bootstrapping loop where test-time compute enables training that improves the model.