Can reasoning happen in latent space without chain of thought?
This explores whether models can do real reasoning inside their hidden activations — without ever writing out the step-by-step "chain of thought" we usually associate with thinking.
This explores whether reasoning can live in a model's internal states rather than in the visible thinking tokens it prints. The short answer from the corpus is yes — and the evidence comes from several directions at once. The most direct case: several architectures (depth-recurrent models, Heima, Coconut) scale up test-time compute by iterating on hidden states instead of generating words, which implies that verbalizing your reasoning is a training habit, not a requirement for actually doing it Can models reason without generating visible thinking tokens?. If reasoning needed language, you couldn't relocate it into silent loops and keep the gains.
The interpretability work makes this concrete at the level of a single knob. Researchers found a reasoning "feature" in the activations and steered it directly — matching or beating chain-of-thought across six model families, with the reasoning mode switching on early in generation and even overriding surface instructions Can we trigger reasoning without explicit chain-of-thought prompts?. That dovetails with a broader finding that base models already carry latent reasoning ability, which five independent techniques (RL steering, critique tuning, decoding tweaks, SAE steering, RLVR) all manage to elicit — post-training selects reasoning that's already there rather than installing it Do base models already contain hidden reasoning ability?. If the capability sits in the weights waiting to be triggered, the chain of thought is more like a switch than the engine.
There's a deeper reframing too: maybe reasoning doesn't have to be token-shaped at all. Meta's Large Concept Model reasons over sentence embeddings in a language-agnostic space before decoding to any language Can reasoning happen at the sentence level instead of tokens?, and energy-based transformers treat inference as gradient-descent energy minimization — a System-2 style of "thinking" that emerges from unsupervised learning with no domain scaffolding and no verbalized steps Can energy minimization unlock reasoning without domain-specific training?. Another line argues you can scale latent reasoning sideways, sampling many parallel hidden trajectories instead of one long serial chain Can reasoning systems scale wider instead of only deeper?. These all describe computation happening in continuous space, where written words never enter the loop.
The interesting tension is what this implies about chain of thought itself. If most of the actual reasoning is latent, then the printed chain may be largely decorative — and the corpus backs that up from the opposite end. Chain of Draft matches full chain-of-thought accuracy using just 7.6% of the tokens, meaning ~92% of a normal chain serves style and documentation, not computation Can minimal reasoning chains match full explanations?. Skeptics push further: chain of thought may be constrained imitation of reasoning's *form* — reproducing familiar templates from training rather than genuine inference, since it breaks down predictably under distribution shift Does chain-of-thought reasoning reveal genuine inference or pattern matching?. Read together, these suggest the visible chain isn't where the thinking happens; it's a readout.
Here's the thing you might not have expected to learn: the question may be backwards. Rather than "can reasoning happen without chain of thought," the corpus increasingly suggests reasoning was *always* happening in latent space, and chain of thought is the externalization we bolted on — sometimes useful for accuracy, sometimes just for show. The frontier work isn't adding hidden reasoning; it's learning to trust it without making the model narrate.
Sources 8 notes
Multiple architectures—depth-recurrent models, Heima, and Coconut—demonstrate that test-time compute scales through hidden state iteration rather than token generation. This suggests verbalization is a training artifact, not a reasoning requirement.
SAE-identified reasoning features can be directly steered to match or exceed chain-of-thought performance across six model families. This reasoning mode activates early in generation and overrides surface-level instructions, suggesting latent reasoning is a fundamental capability independent of explicit prompting.
Five independent mechanisms—RL steering, critique fine-tuning, decoding changes, SAE feature steering, and RLVR—all elicit reasoning already present in base model activations. Post-training selects rather than creates reasoning; the bottleneck is elicitation, not capability acquisition.
Meta's Large Concept Model operates on sentence embeddings rather than tokens, reasoning in a language-agnostic space before decoding to any target language. This hierarchical approach with paragraph-level planning produces more coherent output than flat token generation.
Energy-Based Transformers assign energy values to input-prediction pairs and use gradient descent minimization for inference, yielding 35% higher training scaling rates and 29% more inference-compute gains than Transformer++, while generalizing better on out-of-distribution data without domain-specific scaffolding.
GRAM shows that stochastic latent transitions enabling parallel trajectory sampling sidestep the serial latency cost of depth-only scaling. Width matches token-level parallelism benefits: independent paths sample the solution space without variance inflation.
Chain of Draft achieves equivalent accuracy to standard chain-of-thought on arithmetic, symbolic, and commonsense tasks while using only 7.6% of tokens. The 92.4% of removed tokens served style and documentation, not computation.
CoT works by constraining models to reproduce familiar reasoning patterns from training, not by enabling novel symbolic reasoning. Performance degrades predictably under distribution shifts—the signature of imitation rather than capability emergence.