Why do language models collapse into generic templates?

Naming template collapse is diagnostic; RAGEN-2 also gives a mechanism for why it happens, framed as a signal-to-noise ratio problem in RL. The task-learning signal comes from reward variance across samples of the same prompt. When that within-input reward variance is low — the prompt is too easy, too hard, or otherwise uninformative — the task gradient it produces is weak. With the task signal weak, the regularization terms in the objective (entropy bonuses, KL penalties, and similar) dominate the update, and their effect is to smooth the policy toward generic outputs. The result is that the policy is pushed toward input-agnostic templates: the collapse is not a random drift but the predictable outcome of regularization out-muscling a starved task gradient.

This turns a mysterious failure into a controllable one. Because reward variance is the culprit and it is cheap to compute, the paper proposes SNR-Aware Filtering — selecting high-signal prompts (high reward variance) before each parameter update — which improves performance across tasks, scales, and modalities and slots into existing pipelines. The mechanism generalizes a familiar theme: Does policy entropy collapse limit reasoning performance in RL? describes regularization/optimization forces flattening a policy, and here the same regularization-dominates dynamic produces collapse along a different axis (cross-input distinguishability rather than token entropy). It also motivates curriculum-by-variance: the useful training signal lives at the prompts where reward is neither guaranteed nor impossible, echoing why Why does RL succeed more on some tasks than others? — signal quality, not signal presence, is what trains reasoning.