Can continuous thoughts have tractable likelihoods for sampling and scoring?

Latent reasoning promises a higher-bandwidth alternative to verbalized chain-of-thought: compute in compact continuous states before committing to text. But the vault's existing latent-reasoning thread, since Can models reason without generating visible thinking tokens?, has a quiet liability — most continuous-thought methods throw away the very properties that made textual CoT trainable and steerable: a tractable likelihood, probabilistic sampling, left-to-right generation, KV-cache decoding. Once thoughts are opaque vectors, you can't score a trajectory, sample alternatives, or refine with policy gradients. NF-CoT's contribution is to recover those affordances by modelling continuous thoughts as an autoregressive normalizing flow (TARFlow-style) inside the LLM's own causal stream. An NF head emits continuous-thought positions; the standard LM head emits text positions; both share one causal sequence.

The deeper claim is about modeling status. Text tokens in a CoT are autoregressive, probabilistic, and likelihood-scored — that is why STaR-style training, sampling, and RL refinement work on them. NF-CoT gives continuous thoughts the same status: an explicit distribution over reasoning trajectories with exact likelihood, supporting both supervised likelihood training and policy-gradient refinement in continuous space. This is the missing tractability piece behind the "reasoning need not be verbalized" argument of Can models reason without generating visible thinking steps?, and it complements parameter-side latent scaling such as Can latent thought vectors scale language models beyond parameters? — both add latent structure, but NF-CoT specifically buys likelihood-based control over the latent chain rather than only capacity.

The caveat is scope and provenance. Validation is on code-generation benchmarks only, and the continuous thoughts are distilled from explicit CoT — the flow learns to compress a verbal trace, so it inherits whatever the teacher CoT encoded. The strongest counterargument: if a tractable continuous distribution is achievable only by distilling from text, latent reasoning may remain parasitic on verbalization rather than a genuinely independent reasoning medium. Still, exact likelihood in continuous space is the interface that makes sampling, scoring, and RL on non-verbal thought possible at all, which is a real unlock regardless of where the thoughts originate.