When answers all score the same, the task signal goes quiet and a background pressure silently collapses the model toward one generic response.

This explores why reasoning policies collapse toward narrow, repetitive outputs — and the corpus points to a subtle culprit: when the actual task signal goes quiet, the regularization term becomes the loudest voice in the room. The clearest account is that policies collapse into input-agnostic templates when within-prompt reward variance is low: if every sampled answer to a prompt earns roughly the same reward, the task gradient nearly cancels out, and whatever regularizer is present starts dominating the update — pushing the model toward a generic, safe output that ignores the specifics of the input Why do language models collapse into generic templates?. Entropy doesn't collapse because the model 'decided' to stop exploring; it collapses because the only remaining pressure on the weights is the flattening, mean-reverting pull of regularization.

That mechanism sits underneath the broader empirical picture of entropy collapse as the primary bottleneck in RL for reasoning. Performance follows a tight law, R = -a·exp(H) + b, where gains saturate exactly as policy entropy approaches zero — so anything that accelerates the drain toward zero entropy also caps how good the model can get Does policy entropy collapse limit reasoning performance in RL?. Regularization dominance is one accelerant: on low-variance prompts it spends update after update narrowing the distribution without ever teaching anything, burning through exploratory capacity for no reward gain.

The fix that falls out of this framing is telling. If the problem is low reward variance handing control to the regularizer, then the answer isn't a bigger entropy bonus everywhere — it's picking the *right* prompts. SNR-aware filtering selects high-variance prompts before doing updates, so the task gradient stays strong enough to keep regularization in check, and this recovers performance across tasks and scales Why do language models collapse into generic templates?. This dovetails with the finding that only a minority of tokens — the roughly 20% high-entropy 'forking' decisions — actually carry the RLVR learning signal; when variance collapses, those pivotal forks are exactly what get ironed flat first Do high-entropy tokens drive reasoning model improvements?.

What makes this more than a reasoning-specific quirk is that the same collapse shows up wherever RL narrows behavior. Search agents lose exploration diversity through the identical entropy-collapse mechanism, converging on a few reward-maximizing strategies, while SFT on diverse demonstrations preserves breadth — evidence that the culprit is the RL objective's dynamics, not the reasoning domain Does reinforcement learning squeeze exploration diversity in search agents?. And the collapse you see during training is only half the story: training-time entropy collapse and test-time variance inflation are dual failures of exploration that need structurally different fixes, so taming regularization dominance during training won't automatically stabilize behavior at inference Why do reasoning models fail differently at training versus inference?.

The thing worth walking away with: entropy collapse is often framed as the model 'over-exploiting,' but the sharper diagnosis is that it's the *absence* of a competing signal. When reward variance dries up, regularization doesn't have to win an argument — it wins by default, and the visible symptom is a policy that quietly stops thinking.