Does adding more loops always improve looped language models?

Most framing of looped and recurrent-depth models treats added loops as a monotonic compute dial: more iterations, more effective depth, more reasoning. LoopCoder-v2 — a 7B parallel-loop coder trained from scratch on 18T tokens at several loop counts — breaks that assumption empirically. Two loops deliver broad gains (SWE-bench Verified 43.0 → 64.4), but three or more loops regress. The curve is not a slope toward diminishing returns; it is a sweet spot with a cliff after it.

The diagnostics give the mechanism a location rather than a vibe. Loop 1 establishes the global KV cache every later loop reads from, so it sets a ceiling on the information available downstream. Loop 2 is the principal site of productive refinement — the most coherent hidden-state update, highest inter-loop attention divergence, peak effective rank. Beyond that, loops yield oscillatory updates and reduced representational diversity: the model churns without enriching its state. This complicates the clean efficiency story in Can reasoning be learned during pretraining rather than after? and the "reused computation outperforms added depth" claim of Can looping layers beat adding depth in diffusion models?: reused computation outperforms added depth only up to a count, after which the reuse becomes destructive.

It also sits in productive tension with Can fixed points replace learned halt tokens in reasoning models?. FPRM's premise is that the loop converges to a useful fixed point; LoopCoder-v2 suggests that for some architectures (here, parallel loops with cross-loop position offsets) extra iterations drift into oscillation instead of convergence, and the positional mismatch each loop boundary introduces is part of why. The strongest counterargument is scope: this is one parallel-loop coding architecture trained once per loop count, so the sweet spot may be CLP-specific rather than a universal law of looping. But it is a warning that "loop count" is a design parameter to be selected by a gain–cost analysis, not a free knob to turn up at test time.