When AI keeps improving a pool of ideas, how do you stop it from converging on one answer?

This explores how to keep a pool of candidate ideas or solutions varied while an LLM-driven loop mutates, combines, and selects among them. The corpus has no note on hypothesis populations by name, but it covers the same problem under other labels: evolutionary search, diversity collapse in training, and AI-assisted ideation. Read together, they say diversity has to be protected in three places: the search structure, the model that generates candidates, and the signal that decides which candidates survive.

The search structure is the most direct fix. Mind Evolution runs LLM-written crossover and mutation across an island model, meaning separate subpopulations that evolve in parallel, so one early winner can't take over everything. It solves over 98% of planning tasks and beats best-of-N and sequential revision, all in plain natural language (Can evolutionary search beat sampling and revision at inference time?). Islands only help if the model gives them something different to work with. A model trained toward one best answer hands the search nothing to recombine. Vector Policy Optimization trains models to emit varied but competent solutions instead, which lets evolutionary procedures solve problems that entropy-collapsed policies can't reach at all (Should training maximize diversity when models feed into search?). Its trick is to keep rewards as vectors, one entry per test case, criterion, or persona, rather than summing them into one score. Solutions then spread along the trade-offs the task actually contains, so the diversity comes from the problem rather than from an added randomness knob (Can reward vectors be the hidden source of solution diversity?).

The generator layer matters because diversity tends to leak out during training. Rewarding only final-answer correctness sharpens the model everywhere, and the narrowing spreads even to problems it hasn't solved yet (Does outcome-based RL diversity loss spread across unsolved problems?). Search agents show the same squeeze, while supervised fine-tuning on diverse demonstrations keeps their exploration broad (Does reinforcement learning squeeze exploration diversity in search agents?). Two counter-moves stand out. Step-level critique inside the training loop stops the tails of the distribution from narrowing across self-training rounds (Do critique models improve diversity during training itself?). Training generator and critic agents on different role-specific data avoids the collapse that limits single-agent fine-tuning to about one productive iteration (Can multiple agents stay diverse during training together?). What the selection step rewards also matters. RLHF reduces diversity in code, where correctness rewards convergence, but increases it in creative writing, where distinctiveness is rewarded (Does preference tuning always reduce diversity the same way?). A hypothesis pool needs a selection rule that rewards distinctiveness as well as correctness.

The third protection is to keep the loop from closing on itself. Pure self-improvement stalls partly through diversity collapse, and the methods that work bring in outside anchors such as past model versions, third-party judges, user corrections, or tool feedback (Can models reliably improve themselves without external feedback?). Where the variety comes from also matters. In a preregistered experiment, AI-generated ideas shrank the group's collective diversity, while AI that refined ideas people had already written left it intact. Non-native English speakers contributed the most diversity, and only AI ideation erased that advantage (Does AI assistance homogenize or preserve creative diversity?). So let varied outside sources seed the population and use the model to sharpen candidates rather than to invent them.

Variety alone doesn't help. Multi-agent ideation teams without real domain expertise underperform even a single competent agent, because stimulation without knowledge produces process losses instead of insight (Does cognitive diversity alone improve multi-agent ideation quality?). The goal is competent diversity: keep separate islands, train and score for many good answers rather than one best, and keep some fresh signal from outside the loop.