Can a team of people and AI models get smarter just by averaging their guesses, with no one specializing?
Can ensemble accuracy improve without specialization between team members?
This explores whether a group of predictors (people, models, or agents) can get more accurate together just by combining answers on the same task, without each member taking on a different specialized role, and what that gain actually depends on.
This explores whether a team can beat its members without a division of labor, where everyone attempts the same task and their answers are simply pooled. The corpus says yes, with one condition: the members have to be wrong in different ways. Choudhary and colleagues show that combining human and AI predictions on identical decisions beats either one alone whenever their errors are diverse enough, even when neither is very accurate by itself Can humans and AI improve decisions by predicting together?. The useful ingredient isn't who does what. It's how differently each member errs.
The same logic shows up inside a single model, which is a less obvious place to find it. A generative model trained on data from many imperfect experts can end up outperforming all of them. In effect it absorbs their consensus, and low-temperature sampling pushes its outputs toward that majority view, so idiosyncratic mistakes cancel out Can models trained on many imperfect experts outperform everyone?. So an ensemble doesn't need separate members at runtime. The averaging can be built in during training. Some reinforcement learning work turns the idea into a training goal: SPIRAL rewards a batch of reasoning traces for being useful as a set rather than for each trace being right, and finds that several different mediocre traces beat several near-identical strong ones, because whatever combines them needs real disagreement to arbitrate Can diverse mediocre traces outperform redundant expert traces?.
That also points to how this goes wrong. Diversity can be trained away. RL post-training tends to latch onto one dominant output format from pretraining within the first epoch and suppress the alternatives Does RL training collapse format diversity in pretrained models?. If your ensemble members share a base model and similar post-training, their errors may be more correlated than you'd expect, and most of the ensemble's benefit disappears. Pruning helps from the other side: DyLAN scores each agent's contribution during inference and switches off the ones adding nothing Can multi-agent teams automatically remove their weakest members?. Members that only repeat the others count as uninformative, much like the redundant traces in SPIRAL.
There's a floor, though. Diversity without competence doesn't help and can actively hurt. In multi-agent ideation, cognitively diverse teams lacking real domain expertise did worse than a single competent agent, because the back-and-forth generated noise instead of insight Does cognitive diversity alone improve multi-agent ideation quality?. And for tasks that truly need different skills, parallel work, and independent checking, pooling look-alike generalists hits a ceiling that only role specialization gets past Do single agents always hit organizational limits?.
The takeaway: for prediction and judgment tasks, you can often get accuracy gains without specialization, as long as you deliberately keep the errors uncorrelated. Choose members with different blind spots, and resist training or tuning steps that make them all think alike. Specialization becomes necessary when the work itself has to be split up, not just double-checked.
Sources 7 notes
Choudhary et al. show that aggregating human and AI predictions on identical tasks outperforms either alone whenever their errors differ sufficiently, even if neither achieves high accuracy individually. This requires diversity in how each errs, not specialization.
Models trained on diverse experts converge on consensus behavior that outperforms individuals. Low-temperature sampling concentrates outputs on this majority-voted consensus, denoising uncorrelated biases and errors across the training set.
SPIRAL shifts RL reward from individual traces to sampled sets, optimizing for complementarity rather than per-trace accuracy. Diverse mediocre traces outperform redundant strong ones because aggregators need raw material to arbitrate, not confirmation.
Controlled experiments show RL consistently amplifies one format distribution from pretraining within the first epoch while collapsing alternatives. The winning format depends on model scale, not necessarily performance, and is largely hidden when starting from proprietary pretrained models.
DyLAN's three-step importance scoring mechanism (propagation, aggregation, selection) quantifies individual agent contributions and automatically removes uninformative agents during inference, optimizing team composition without task-specific tuning.
Show all 7 sources
Multi-agent teams substantially outperform solo ideation, but only when members possess genuine senior knowledge. Diverse teams without expertise underperform even a single competent agent, because cognitive stimulation without expertise triggers process losses instead of insight.
Research shows that real-world tasks requiring heterogeneous expertise, parallel execution, and independent verification exceed what any single agent loop can organize. Graph-based system abstractions are needed to distribute intelligence across specialized agents.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Towards a Science of Scaling Agent Systems
- Human-AI Ensembles: When Can They Work?
- Self-Organizing Agent Teams Learn to Reason Together
- Worse Together: How Performance Breaks Down in Multi-User Multi-Agent Teams
- How we built our multi-agent research system
- The Art of Scaling Reinforcement Learning Compute for LLMs
- Sharpening Tax in Post-Training
- Planted in Pretraining, Swayed by Finetuning: A Case Study on the Origins of Cognitive Biases in LLMs