Combining lots of AI predictions boosts accuracy mainly by averaging out noise, not by blending truly different viewpoints — why?
Why do aggregated AI predictions gain accuracy mainly from scale rather than diversity?
This explores why combining many AI predictions (ensembles, crowds of models, majority votes) improves accuracy mostly by adding more samples rather than by adding genuinely different viewpoints. The corpus suggests the reason is simple: AI models are far less different from one another than they look.
This explores why combining many AI predictions improves accuracy mostly by adding more samples rather than by adding genuinely different viewpoints. The corpus has no single study that tests 'scale vs. diversity' head-to-head. It does contain a clear chain of evidence that explains the pattern: the diversity that would make aggregation powerful is mostly missing from AI models to begin with.
Start with the theory of why aggregation works at all. Combining predictions helps only when the predictors make different mistakes. If everyone errs the same way, averaging just repeats the error more confidently. Choudhary and colleagues make this explicit for human-AI pairs: their combined prediction beats either one alone whenever their errors differ enough, even if neither is very accurate on its own Can humans and AI improve decisions by predicting together?. The point is that what you need is diversity in *how each one errs*, not raw skill. A related result shows that generative models can outperform the imperfect experts they were trained on through a kind of built-in majority vote. Low-temperature sampling settles on the consensus answer and cancels out errors that are uncorrelated across the training data Can models trained on many imperfect experts outperform everyone?.
The catch is that AI models rarely supply that kind of diversity. A study of 70+ models answering 26,000 open-ended prompts found an 'Artificial Hivemind': different models from different labs independently produce strikingly similar or even identical answers, because they share training data and alignment methods Do different AI models actually produce diverse outputs?. Swapping one model for another therefore adds less independent signal than you would expect. Another note frames the same problem at the level of ideas: AI multiplies claims without multiplying points of view, so a thousand AI-written articles may amount to roughly one perspective Does AI generate diverse claims or diverse perspectives?.
Post-training makes this worse. RL fine-tuning quickly locks onto one dominant output format from pretraining and suppresses the alternatives Does RL training collapse format diversity in pretrained models?. On agentic tasks, post-training sharpens results on easy cases while wiping out rare-but-reachable solutions. Given enough attempts, plain base models end up covering more of the solution space than their polished counterparts Do base models find more solutions than post-trained ones?. The effect does depend on the domain: preference tuning narrows variety in code but can increase it in creative writing Does preference tuning always reduce diversity the same way?. So if your ensemble members are all aligned models, they have already been pushed toward the same answers. What's left to gain is mostly the statistical benefit of drawing more samples.
The less obvious takeaway: diversity isn't useless, it just has to be real and grounded. In multi-agent brainstorming, teams with varied thinking styles beat solo agents, but only when each member brings real domain expertise. Diverse but shallow teams do worse than a single competent agent Does cognitive diversity alone improve multi-agent ideation quality?. Read together, these notes suggest the gains don't come from scale winning over diversity. They come from scale because current AI rarely offers diversity worth aggregating. The more promising sources of uncorrelated errors are combining humans with AI, using base models alongside aligned ones, or using agents with genuinely different expertise.
Sources 8 notes
Choudhary et al. show that aggregating human and AI predictions on identical tasks outperforms either alone whenever their errors differ sufficiently, even if neither achieves high accuracy individually. This requires diversity in how each errs, not specialization.
Models trained on diverse experts converge on consensus behavior that outperforms individuals. Low-temperature sampling concentrates outputs on this majority-voted consensus, denoising uncorrelated biases and errors across the training set.
INFINITY-CHAT analyzed 70+ models across 26K open-ended queries and found an "Artificial Hivemind" effect: models independently generate strikingly similar or identical responses due to overlapping training data and alignment procedures, undermining the diversity benefits of model ensembles.
Large language models generate numerous well-formed claims by following probabilistic patterns in training data, not by exploring competing argumentative positions. This produces volume without perspectival diversity—a thousand AI articles often represent approximately one viewpoint.
Controlled experiments show RL consistently amplifies one format distribution from pretraining within the first epoch while collapsing alternatives. The winning format depends on model scale, not necessarily performance, and is largely hidden when starting from proprietary pretrained models.
Show all 8 sources
Across 14 model pairs and three agentic benchmarks, base models equipped with only relaxed system prompts eventually surpass post-trained counterparts in pass@K coverage as rollout budget grows. Post-training bimodalizes task outcomes, sharpening performance on easy cases while eliminating rare-but-reachable solutions.
RLHF reduces lexical-syntactic diversity in code generation but increases it in creative writing. The direction depends on what each domain incentivizes: code rewards convergence toward correct solutions, while creative writing rewards stylistic distinctiveness.
Multi-agent teams substantially outperform solo ideation, but only when members possess genuine senior knowledge. Diverse teams without expertise underperform even a single competent agent, because cognitive stimulation without expertise triggers process losses instead of insight.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- NoveltyBench: Evaluating Language Models for Humanlike Diversity
- Human diversity fuels collective creativity that large language models cannot simulate or sustain
- Generative Artificial Intelligence and Evaluating Strategic Decisions
- Sharpening Tax in Post-Training
- On Epistemic Diversity in Large Language Models
- Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond)
- ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs
- What and Whose Knowledge? Measuring Epistemic Diversity in Large Language Models