Can entropy metrics detect when reasoning becomes formulaic?

In multi-turn agentic RL, researchers monitor entropy as a proxy for reasoning stability. RAGEN-2 shows this proxy has a blind spot. Entropy measures only within-input diversity — how varied the sampled reasoning is for a single input — and therefore cannot tell whether reasoning actually responds to different inputs. A model can maintain stable, high conditional entropy while its reasoning has drifted to fixed templates: fluent boilerplate that looks diverse within any one input but is effectively the same across all of them. The authors name this template collapse and call it invisible to entropy and every existing metric.

The fix is a proper decomposition: reasoning quality splits into within-input diversity (entropy) and cross-input distinguishability, captured by the mutual information (MI) between inputs and reasoning. Across diverse tasks, MI correlates with final performance much more strongly than entropy, making it the more reliable online proxy. This sharpens the RLVR-dynamics literature: it means Does policy entropy collapse limit reasoning performance in RL? is only half the picture — entropy can stay healthy while reasoning silently decouples from the input, so a stable entropy curve is not evidence of stable reasoning. The MI framing also rhymes with Can models learn behavioral principles without preference labels?: input-to-reasoning MI is the diagnostic dual of the objective SAMI maximizes — both treat conditional dependence, not raw diversity, as the quantity that actually matters.