As AI models get bigger, does thinking silently fall further behind writing out its steps — or catch up?
Does the latent-explicit gap widen beyond 3B parameters on reasoning tasks?
This explores whether latent reasoning (thinking in hidden states) falls further behind explicit chain-of-thought (writing the steps out) as models grow past 3B parameters, or whether the gap stays closed.
This explores whether latent reasoning falls further behind explicit chain-of-thought as models grow past 3B parameters. The corpus can't answer that directly. Its only size-specific result sits at 3B, and nothing compares latent and explicit reasoning at 7B, 70B or beyond.
The 3B result is encouraging but narrow. LOTUS, a looped Transformer trained with gold chain-of-thought supervision at each latent position, Can latent reasoning close the scaling gap with explicit chain-of-thought? closes the latent-to-explicit gap on GSM8K and cuts thought-phase latency by 2.5x to 6.9x. That is one benchmark, one model size, and a recipe that borrows supervision from explicit reasoning. The gap looks more like a training-recipe problem than a fixed size penalty, but a single point can't show a trend. A 150M-parameter model with in-context demonstrations and recurrent latent computation reached 29.5% pass@2 on ARC-AGI-1 at $0.0007 per task Can latent reasoning match chain-of-thought cost efficiency without verbalizing?. That suggests small size isn't the binding limit for latent methods. It's a different task and a cost-accuracy comparison, though, not a matched gap measurement.
Several notes point indirectly toward the gap not widening, and this is inference, not measurement. Explicit chain-of-thought has its own scale dependence: the best length follows an inverted U, and more capable models prefer shorter chains, with RL training drifting toward brevity as models improve Why does chain of thought accuracy eventually decline with length?. If stronger models need less verbalization, explicit reasoning's edge should shrink with scale. Separately, depth-recurrent models, Heima and Coconut scale test-time compute through hidden-state iteration Can models reason without generating visible thinking tokens?, which fits the view that spelling out steps is a training habit, not a requirement. And if base models already hold reasoning ability that minimal training unlocks Do base models already contain hidden reasoning ability?, then bigger bases have more to elicit, and the question becomes how well each method draws it out.
The other results suggest that what limits reasoning may be less about size and more about what the model has seen. Chain-of-thought degrades predictably under distribution shift Does chain-of-thought reasoning actually generalize beyond training data?. Reasoning failures track how unfamiliar an instance is, not how complex the task is Do language models fail at reasoning due to complexity or novelty?. Models trained on deliberately corrupted traces do about as well as those trained on correct ones, which suggests traces work as computational scaffolding, not as meaningful steps Do reasoning traces need to be semantically correct?. If explicit tokens are mostly scaffolding, a latent scaffold could plausibly replace them at any size. That would make the training data matter more than parameter count.
The missing experiment is a matched latent-versus-explicit comparison across several model sizes on the same reasoning tasks, and the collection doesn't yet have one. The direct answer is that no evidence here shows the gap widening past 3B. The indirect signals lean toward it staying flat or shrinking, but that is a hypothesis to test, not a finding.
Sources 8 notes
LOTUS, a looped Transformer trained with gold CoT supervision at each latent position, closes the latent-to-explicit gap on GSM8K and cuts thought-phase latency by 2.5× to 6.9×. Latent reasoning need not be verbalized token-by-token to remain legible and competitive.
A 150M-parameter model combining in-context demonstrations with iterative latent computation reached 29.5% pass@2 on ARC-AGI-1 at $0.0007 per task, surpassing previously reported cost-accuracy tradeoffs. The approach separates learning (via demonstrations updating recurrent memory) from reasoning (via iteration in hidden space) without generating intermediate tokens.
Task accuracy peaks at intermediate CoT length, with optimal length increasing alongside task difficulty but decreasing with model capability. RL training naturally gravitates toward shorter chains as models improve, revealing that simplicity emerges from reward signals rather than explicit training.
Multiple architectures—depth-recurrent models, Heima, and Coconut—demonstrate that test-time compute scales through hidden state iteration rather than token generation. This suggests verbalization is a training artifact, not a reasoning requirement.
Five independent mechanisms—RL steering, critique fine-tuning, decoding changes, SAE feature steering, and RLVR—all elicit reasoning already present in base model activations. Post-training selects rather than creates reasoning; the bottleneck is elicitation, not capability acquisition.
Show all 8 sources
DataAlchemy experiments show CoT fails systematically under distributional shifts in task, length, and format. Models produce fluent but logically inconsistent reasoning — imitating reasoning form without valid underlying logic.
LRMs don't break at complexity thresholds but at instance-novelty boundaries. Models fit instance-based patterns rather than generalizable algorithms, so any reasoning chain succeeds if trained on similar instances, regardless of length.
Models trained on systematically irrelevant traces maintain solution accuracy and sometimes improve out-of-distribution generalization, suggesting traces function as computational scaffolding rather than meaningful reasoning steps.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach
- Hierarchical Reasoning Model
- Reasoning Beyond Chain-of-Thought: A Latent Computational Mode in Large Language Models
- When More is Less: Understanding Chain-of-Thought Length in LLMs
- Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers
- Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens
- Beyond Semantics: The Unreasonable Effectiveness of Reasonless Intermediate Tokens
- Break the Chain: Large Language Models Can be Shortcut Reasoners