Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers

Language models typically reason via explicit chain-of-thought (CoT), generating intermediate steps token-by-token. Latent CoT offers an alternative: it performs multi-step reasoning in the model’s hidden states, replacing decoded tokens with continuous representations for greater efficiency. However, existing latent CoT methods underperform explicit CoT beyond 1B parameters, and the gap widens with scale. Looped, or recurrent-depth, Transformers, which reuse their weights to increase computation depth without adding parameters, are a natural fit for latent reasoning. We therefore ask whether looped Transformers can bridge this gap. We answer affirmatively with a simple recipe: a looped padded Transformer that processes K latent blocks in parallel for R iterations, with a cross-entropy loss on each latent position’s gold CoT-step token, similar to explicit CoT supervision. We instantiate it as LOTUS (Looped Transformers with parallel supervision on latents). LOTUS is, to our knowledge, the first latent-CoT method to bridge the gap to explicit CoT at the 3B scale, while cutting thought-phase latency by 2.5ˆ– 6.9ˆ from compact math expressions to natural language.

Introduction. Scaling inference compute, i.e., letting a model “think” before it answers, has become a dominant lever for increasing language model capabilities, with stronger performance now coming from longer reasoning chains rather than from model size alone [DeepSeek-AI, 2025, OpenAI, 2026]. Chain-ofthought (CoT) reasoning [Wei et al., 2022], where the model emits intermediate reasoning steps, drives this trend. However, since each token must be decoded sequentially, generating a CoT of length N takes N sequential model evaluations, making reasoning costly. Latent reasoning aims to achieve the same at a fraction of the cost: it carries out the intermediate computation in continuous hidden states rather than decoded tokens, condensing many steps into fewer model evaluations. On small backbones such as GPT-2 [Radford et al.], latent methods [Hao et al., 2025, Shen et al., 2025, Wei et al., 2025] already match CoT accuracy.

Discussion / Conclusion. We introduce LOTUS, showing that latent reasoning can approach the performance of explicit CoT by supervising a looped padded Transformer in parallel against the gold CoT tokens under the simple cross-entropy objective. On LLAMA-3.2-3B-INSTRUCT, LOTUS bridges the in-domain gap to explicit CoT on GSM8K, surpasses CoT on the out-of-domain average, and cuts thought-phase latency by 2.5ˆ. Ablations show the looped backbone, parallel gold CoT supervision, and sufficient block width and loop depth are each necessary. The latent representation analysis further shows the latents are transparent: the gold CoT is recoverable from them by a direct readout, they place graded probability on unseen but valid reasoning chains rather than a single memorized trace, and the step and answer losses contribute complementary structure. Limitations. We follow prior latent reasoning work [Hao et al., 2025, Shen et al., 2025, Wei et al., 2025] and evaluate on math benchmarks. Whether the recipe transfers to other domains remains an open direction for future work.