Can models trained on many imperfect experts outperform everyone?
Can generative models trained on diverse, biased experts achieve better performance than any individual contributor? This explores whether aggregating diverse perspectives during training acts as implicit denoising.
The Transcendence paper formalizes a surprising property: generative models trained on many experts with diverse capacities and biases can outperform any single expert. The mechanism is implicit majority voting. When trained on diverse human players (chess), the model's cross-entropy optimization converges on the consensus behavior — which, by the wisdom-of-the-crowd effect, is often better than any individual contributor.
Low-temperature sampling is the key enabler. At low temperature, the model's output distribution concentrates on its highest-probability predictions — the consensus. This is formally equivalent to a majority vote. The advantage is primarily due to performing much better on a small subset of states — likely the critical, outcome-determining positions where individual human biases diverge most and the crowd wisdom is most valuable.
Diversity in the training data is a necessary condition. Without diversity, there is no denoising — a model trained on clones of one expert can only approach that expert's level. The practical conditions for transcendence: (1) diverse training sources with different biases, (2) a task where individual biases are uncorrelated (so they cancel under aggregation), and (3) low-temperature decoding to extract the consensus.
This connects to but is distinct from Why does majority voting outperform more complex inference methods?. That note describes inference-time majority voting over multiple samples from one model. Transcendence describes training-time majority voting implicitly encoded in a single model's weights through diverse training data. The mechanism is analogous — aggregation denoises — but operates at different timescales.
The implication for LLM training is provocative: the "average" of many imperfect human demonstrations may be better than any individual human demonstration, provided the imperfections are diverse rather than correlated. This challenges the assumption that training data quality should be maximized per-example; quantity and diversity of perspectives may matter as much as individual quality.
Inquiring lines that read this note 45
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
What systematic biases do LLM judges exhibit in evaluation?- How does same-author bias interact with the four adversarial judge biases already documented?
- Does ensembling smaller judges reduce bias more effectively than single large judges?
- Why does diversity without expertise produce worse results than a single capable agent?
- Can ensemble accuracy improve without specialization between team members?
- Can diverse expert demonstrations exceed the knowledge of any single expert?
- When do aggregated imperfect demonstrations fail to outperform the best expert?
- Does debiasing training data actually solve the bias problem in machine learning?
- Why does low temperature sampling extract consensus from diverse training data?
- How does majority voting fail when reasoning samples lack genuine diversity?
- Why do self-consistency methods fail where pretraining bias is strongest?
- What signals detect when consensus training is silently degrading performance?
- What happens when majority voting converges to a single answer?
- What compute costs does majority-vote consensus sampling add versus supervised training?
- What mechanisms let generative models escape collapse through majority voting?
- Can aggregation cancel out independent errors but not shared training-data biases?
- What conditions make training diversity better than individual expert quality?
- How does mutual shaping through diverse training compare to population-level diversity effects?
- Can population-level distributions shift usefully even when individual prediction fails?
- Does disjoint family diversity actually cancel model-specific bias in evaluation?
- Why does diversity in training data enable denoising rather than reinforce shared biases?
- Can temperature-adaptive sampling reduce the sharpening tax during training?
- Why does diversity of training cases matter more than raw dataset size?
- Why do aggregated AI predictions gain accuracy mainly from scale rather than diversity?
- Can individually accurate agents still fail at population-level representation?
- Can small directional biases add up to meaningful population effects?
- Why does prediction error overlap matter more than individual performance?
- What happens to human decision authority when ensembles prioritize error diversity?
- How does correctness emergence occur when no expert initially solved the task?
- What happens when models train on feedback from their own generations?
- How does generative intelligence differ from the bounded intelligence of individual experts?
- Does generative AI narrow performance gaps between different professional backgrounds?
- When does generative AI reduce versus amplify existing skill differences?
- How does generativity through quasi-random sampling produce constrained creativity?
- How much noise comes from rater idiosyncrasy versus selection bias?
- How do ensemble methods reduce bias in automated evaluation?
Related concepts in this collection 5
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Why does majority voting outperform more complex inference methods?
Simple majority voting across independent samples often matches or beats sophisticated alternatives like Best-of-N and sequential revision. What makes this basic approach so hard to beat for reasoning models?
inference-time voting analog; this is the training-time version
-
Does voting discard useful reasoning from losing chains?
When multiple reasoning chains compete through majority voting, intermediate steps from non-winning chains are discarded. Could extracting and mixing those intermediate facts improve both the final answer and our ability to understand the reasoning?
shows limits of pure voting; transcendence may have similar limits
-
Does training on AI-generated content permanently degrade model quality?
When generative models train on outputs from previous models, do the resulting models lose rare patterns permanently? The question matters because future training data will inevitably contain synthetic content.
counterpoint: while diversity enables transcendence, synthetic data collapses diversity
-
Can generative and discriminative models reach agreement?
Generative and discriminative decoding often produce conflicting answers. Can a game-theoretic framework force these two complementary procedures to reconcile their predictions into a single, more reliable output?
related consensus mechanism: transcendence achieves consensus across diverse training experts at training time, while Consensus Game achieves consensus between generative and discriminative decoding modes at inference time; both extract a signal more reliable than any single perspective
-
Can a quorum of validators really provide independent judgment?
If multiple validators share training data, prompts, evidence sources, or infrastructure, their agreement may reflect shared causes rather than independent confirmation. This could make quorum-based systems less reliable than they appear.
the deployment-time inverse of the uncorrelated-biases condition: validators chosen at inference time may share weights, prompts, retrieval and evidence, so a vote among them need not denoise anything; that note reports no measured correlation
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Transcendence: Generative Models Can Outperform The Experts That Train Them
- Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data
- Escaping Model Collapse via Synthetic Data Verification: Near-term Improvements and Long-term Convergence
- Human-AI Ensembles: When Can They Work?
- Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond)
- Generative Artificial Intelligence and Evaluating Strategic Decisions
- Selective Reflection-Tuning: Student-Selected Data Recycling for LLM Instruction-Tuning
- Planted in Pretraining, Swayed by Finetuning: A Case Study on the Origins of Cognitive Biases in LLMs
Original note title
generative models transcend their training experts through implicit majority voting that denoises diverse human biases