How do looped language models actually improve reasoning in depth?

Looping an LLM's layers in the latent dimension improves reasoning, but it has been unclear how the internal dynamics differ from a standard feedforward model. This mechanistic analysis answers through the lens of stages of inference — the idea that LLM computation decomposes into distinct computational stages.

The core result is geometric. For many looped models, each layer in the cycle converges to a distinct fixed point, so the recurrent block follows a consistent cyclic trajectory in latent space. As those fixed points are reached, attention-head behavior stabilizes, producing constant behavior across recurrences. And empirically the recurrent blocks learn stages of inference that closely mirror feedforward models — repeating those stages in depth with each iteration. This appears to be emergent: it shows up even when training does not explicitly encourage it. The repeated application of a shared block necessarily implies one of two regimes — either the block's contribution vanishes asymptotically, or it traces a constant cyclic trajectory.

The implication that matters: recurrent depth is learned re-application of computation, not the discovery of genuinely new computation per loop. The loop re-runs the same inferential stages rather than adding qualitatively different ones. This is the mechanistic complement to Can looping layers beat adding depth in diffusion models?: it explains why reused computation can match or beat added depth — the network was re-enacting stages anyway, and looping makes that reuse explicit and parameter-free. Recurrent block size, input injection, and normalization govern whether these cyclic fixed points emerge and stay stable.