When AI is trained to get answers right, why does it end up thinking through every problem the same way?

This explores why ordinary RL — which scores each reasoning trace on whether it individually reaches the right answer — tends to funnel a model's outputs into a handful of near-identical paths rather than a spread of different attempts. The clearest account of the mechanism comes from set-level reinforcement learning: when the reward lands on each trace separately, the optimizer's only incentive is to make every trace look more like the current best-known correct trace. Diversity has no reward attached to it, so it decays, and you end up with a batch of confident traces that all say the same thing. The alternative that exposes the problem is to reward a *set* of traces for being collectively useful — under that objective, several mediocre-but-different traces beat several redundant strong ones, because an aggregator needs varied raw material to arbitrate rather than repeated confirmation Can diverse mediocre traces outperform redundant expert traces?.

Why does per-trace optimization pull toward redundancy specifically? Part of the answer is that RL post-training mostly isn't teaching new reasoning — it's tuning *when* to deploy reasoning the base model already contains. If the capability is latent and RL is optimizing deployment timing, then the fastest way to raise average reward is to converge hard on whichever pre-existing path scores well and stop sampling the others Does RL post-training create reasoning or just deploy it?. That collapse toward one route is visible at decoding time too: reasoning models 'wander' and then switch away from promising paths prematurely, behaving like tourists who abandon a good trail rather than scientists who follow it — a structural disorganization, not a compute shortage, which suggests the good paths exist but the training pressure isn't rewarding sticking with diverse ones Why do reasoning models abandon promising solution paths?.

There's also a subtler cost hiding inside 'correct' traces. Even traces that reach the right answer can carry redundant tails — reasoning that continues after the answer is already settled — and that post-conclusion padding actively *harms* downstream fine-tuning, more than removing an equally long random chunk would Does every correct chain-of-thought trace improve fine-tuning?. Correctness-only rewards can't see this, because they look at the final answer, not the shape of the path. The same blind spot shows up when you realize corrupted or irrelevant traces can teach nearly as well as clean ones — traces often function as computational scaffolding rather than meaningful logic — so an objective that grades only the endpoint has no way to prefer a genuinely different route over a redundant one Do reasoning traces need to be semantically correct?.

The fixes in the corpus all work by putting reward back onto diversity or on the *process* rather than the endpoint. Asymmetric trajectory filtering keeps positive traces clean but deliberately preserves varied failures as negative signal, letting a 14B model reach frontier math performance precisely because it doesn't discard the diversity in its wrong answers Why do correct code trajectories teach models to tolerate errors?. Process-level verification catches errors mid-trace that final-answer scoring misses entirely — raising success from 32% to 87% — which reframes reliability as checking how you got there, not just where you landed Where do reasoning agents actually fail during long traces?. And difficulty-aware schemes hand partial solution traces as guidance only where reward is otherwise sparse, converting problems that would collapse into a single memorized path into genuine exploration signal Can adaptive guidance from solution traces reduce reward sparsity in RL?. The thread across all of them: redundancy is what you get when the reward function can only see individual endpoints — the moment you reward the collective, the process, or preserved variety, the traces stop collapsing.