How do pre-norm layers enable reliable fixed-point halting signals?

This reads the question as really being about *fixed-point convergence as a halting signal* — the idea that a model running the same layer over and over can decide it's done when its internal state stops changing, rather than waiting for a special 'halt' token it was trained to emit. On the literal premise (that pre-norm layers are the enabling trick), the corpus doesn't make that argument; what it does have is a sharp result on why fixed-point detection beats the alternatives. Worth flagging that gap up front so you can recalibrate.

The load-bearing note here is Can fixed points replace learned halt tokens in reasoning models?. A looped transformer reuses the same weights across many passes, so its hidden state traces a trajectory. FPRM's finding is that watching for when that trajectory settles into a fixed point — successive passes barely move the state — tracks the moment accuracy saturates more faithfully than a learned halt token does, and without needing a special training regime to teach it when to stop. The 'reliability' you're asking about comes from the geometry of convergence itself being a calibrated proxy for 'I've thought enough,' not from a token the model might emit too early or too late.

Why would anyone want looped depth in the first place? Because fixed-depth transformers hit a ceiling. Can recurrent hierarchies achieve reasoning that transformers cannot? shows a 27M-parameter recurrent model nailing Sudoku and mazes that chain-of-thought models fail, precisely by escaping the complexity ceiling that constrains a fixed stack of layers. Recurrence buys you variable, problem-dependent compute — and once compute is variable, you *need* a halting rule, which is exactly the slot fixed-point detection fills.

But there's a caution the corpus raises about the word 'reliable.' Does setting temperature to zero actually make LLM outputs reliable? makes the point that consistency is not the same as correctness — a model can converge cleanly and repeatedly to the same answer that is still wrong. And Can models be smart without organized internal structure? shows internal state can look organized (linearly decodable, stable) while being fractured underneath. So a fixed point tells you the computation has *settled*, not that it settled on something true. The halting signal is a compute-calibration tool, not a correctness guarantee.

The thing you might not have expected to learn: the interesting design move isn't normalization at all — it's that recurrence reframes 'when to stop' from a learned behavior (a halt token you have to train and trust) into an observable physical property of the model's own dynamics. That's a cleaner, more falsifiable signal, and it's why fixed points are winning the halting debate.