If AI learns from AI-made data, can it still end up smarter than every teacher it learned from?
Can synthetic data diversity preserve the transcendence effect or does it collapse?
This explores whether the effect where a model trained on many imperfect experts outperforms every one of them (transcendence) can survive when the training data is synthetic, and whether the diversity that effect depends on gets preserved or lost.
This explores whether transcendence, where a model trained on many imperfect experts beats each of them, survives synthetic data, and whether the diversity it needs can be kept or gets lost. The corpus never runs that exact experiment. But the pieces point to an answer: it can survive only if diversity is deliberately built in, and the default synthetic pipeline tends to collapse it. Transcendence works like an implicit majority vote. Models trained on diverse experts settle on the consensus, and low-temperature sampling concentrates outputs there, so the experts' uncorrelated mistakes cancel out (Can models trained on many imperfect experts outperform everyone?). Diversity is the raw material. With no disagreement among the experts, there is nothing to vote across.
The default synthetic path loses that raw material. Self-improvement loops degrade through irreversible diversity loss, partly because evaluations fold quality, diversity, and complexity into one score. Quality helps within the training distribution, while diversity is what helps outside it (How do quality, diversity, and complexity affect synthetic data differently?). The same pattern shows up downstream. When most of a retrieval corpus becomes synthetic, results homogenize while answer accuracy stays high, so the decay is hidden (Does synthetic content in search results hide ecosystem decay?). The generators are also often pre-squeezed. RL post-training converges on one dominant pretraining format (Does RL training collapse format diversity in pretrained models?) and narrows exploration in search agents, while SFT on diverse demonstrations keeps it broad (Does reinforcement learning squeeze exploration diversity in search agents?). Even that varies by domain. Preference tuning reduces lexical diversity in code but increases it in creative writing (Does preference tuning always reduce diversity the same way?).
There is a subtler problem, and this part is my inference from the notes, not something the corpus tests. Majority voting only cancels errors that are uncorrelated. Synthetic data from many LLM sources may not be independent. One study of 70+ models on 26K open-ended queries found they independently produce strikingly similar outputs, because their training data and alignment procedures overlap (Do different AI models actually produce diverse outputs?). If synthetic experts share the same blind spots, the vote amplifies their mistakes rather than cancelling them. What matters is whether the sources err differently, not how many there are.
Diversity can be engineered instead of hoped for. Simula builds a taxonomy for global coverage and then refines locally for diversity and complexity, with no seed examples (Can we generate synthetic data without any seed examples?). Realistic dialogues come from multiplying subtopic, Big Five persona, and contextual characteristics (Can synthetic dialogues become realistic through layered diversity?). TarGEN seeds generation with atomic task elements rather than full examples (Can synthetic data replace seed examples in task generation?). Structured variation like this is closer to having many different experts than resampling one model. The corpus also warns that no single recipe works everywhere, since the effect of diversity depends on domain, model, and scale (What makes synthetic data work across different domains and models?).
The twist is that transcendence is itself a kind of collapse, because low-temperature sampling deliberately concentrates on the consensus. The useful line is where the collapse happens. Diversity has to live in the training data, and narrowing is only wanted at sampling time. Synthetic loops break the effect by narrowing the training data itself.
Sources 11 notes
Models trained on diverse experts converge on consensus behavior that outperforms individuals. Low-temperature sampling concentrates outputs on this majority-voted consensus, denoising uncorrelated biases and errors across the training set.
Quality drives in-distribution generalization, diversity enables out-of-distribution generalization, and complexity strengthens both. Current evaluation methods collapse these into a single quality metric, causing self-improvement loops to degrade through irreversible diversity loss.
When 67% of a corpus becomes synthetic, over 80% of retrieved results shift to synthetic sources while answer accuracy remains high, masking the loss of source diversity. This creates fragility: high accuracy resting on a monoculture collapses when that monoculture is poisoned.
Controlled experiments show RL consistently amplifies one format distribution from pretraining within the first epoch while collapsing alternatives. The winning format depends on model scale, not necessarily performance, and is largely hidden when starting from proprietary pretrained models.
RL training compresses behavioral diversity in search agents through the same entropy collapse mechanism documented in reasoning—policies converge on narrow reward-maximizing strategies. SFT on diverse demonstrations preserves exploration breadth, suggesting diversity-preservation techniques are essential for RL search scaling.
Show all 11 sources
RLHF reduces lexical-syntactic diversity in code generation but increases it in creative writing. The direction depends on what each domain incentivizes: code rewards convergence toward correct solutions, while creative writing rewards stylistic distinctiveness.
INFINITY-CHAT analyzed 70+ models across 26K open-ended queries and found an "Artificial Hivemind" effect: models independently generate strikingly similar or identical responses due to overlapping training data and alignment procedures, undermining the diversity benefits of model ensembles.
Simula separates global coverage from local diversity, using taxonomy construction for coverage and agentic refinement for complexity. This architecture makes all three desiderata—quality, diversity, complexity—controllable simultaneously without requiring seed data.
Research shows that realistic synthetic dialogues require three multiplicative layers: subtopic specificity, Big Five persona variation, and 11 contextual characteristics via Chain of Thought reasoning. This structured approach captures 90.48% of in-domain dialogue performance.
TarGEN generates synthetic data using atomic task elements (instance seeds) instead of full input-output examples, achieving 1-3 point improvements on SuperGLUE tasks. The approach works by constraining label generation after seeding inputs, enabling data creation for domains with no prior examples.
Research shows no single optimal recipe for synthetic data generation. The impact of data properties like complexity and diversity varies by domain, model, use case, and scale, making explainable, flexible control more valuable than one-size-fits-all methods.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Orchestrating Synthetic Data with Reasoning
- Reasoning-Driven Synthetic Data Generation and Evaluation
- Scaling Synthetic Data Creation with 1,000,000,000 Personas
- A Little Human Data Goes A Long Way
- Surveying the Effects of Quality, Diversity, and Complexity in Synthetic Data From Large Language Models
- Evaluating the Diversity and Quality of LLM Generated Content
- Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond)
- NoveltyBench: Evaluating Language Models for Humanlike Diversity