Can an AI think in hidden numbers instead of words and still leave a trail you can actually check?
Can latent reasoning stay readable without explicit token-by-token decoding?
This explores whether a model can do its thinking in hidden numerical states instead of written-out words, and still leave a trail a person could inspect, rather than being a black box that only shows its final answer.
This explores whether a model can think in hidden numerical states instead of written-out words and still leave something a person can inspect. The collection has no paper that solves this head-on. Its notes do suggest the question is framed slightly wrong: the token-by-token text was never as readable as it looks, and hidden computation is less hidden than it sounds.
First, what you give up by dropping the words. Latent reasoning works and is cheap. Depth-recurrent models, Heima and Coconut all scale test-time compute by iterating on hidden states instead of generating tokens (Can models reason without generating visible thinking tokens?). A 150M-parameter model reached 29.5% on ARC-AGI-1 at $0.0007 per task with no intermediate tokens at all (Can latent reasoning match chain-of-thought cost efficiency without verbalizing?). Looped architectures at least give a visible signal: when the state stops changing, the model halts (Can models learn by looping instead of growing larger?). That tells you when the model is done, not what it thought. Pure latent reasoning gets efficiency and pays for it in inspectability.
The written trail is weaker evidence than we assume, though. Models trained on deliberately corrupted, irrelevant reasoning traces do about as well as models trained on correct ones, so traces often work as computational scaffolding, not as an honest explanation (Do reasoning traces need to be semantically correct?). Chain of Draft matches full chain-of-thought accuracy with 7.6% of the tokens, and the other 92.4% turned out to be style and documentation (Can minimal reasoning chains match full explanations?). The reverse also holds. A logit-lens study found that a model trained to emit filler tokens computes the right answer in layers 1-3, then suppresses it in the final layers to produce format-compliant filler. The reasoning stays fully recoverable from lower-ranked token predictions (Do transformers hide reasoning before producing filler tokens?). So readability can come from reading the internals with the right instrument, and the output text can be the less honest witness. Reasoning that is already present in a base model's activations can likewise be located and steered with sparse-autoencoder features (Do base models already contain hidden reasoning ability?).
Two designs sit between fully verbal and fully hidden. Meta's Large Concept Model reasons over sentence embeddings and decodes into words only at the end, so the readable unit is a whole sentence-sized idea, decoded on demand and in any language (Can reasoning happen at the sentence level instead of tokens?). Diffusion language models keep the reasoning as text but refine it in place alongside the answer, not left to right. Answer confidence settles early while the reasoning keeps sharpening, which lets you peek at intermediate states and exit early to save about half the compute (Can reasoning and answers be generated separately in language models?).
So the answer is yes in part, if you change what readable means. It stops being the model wrote out its steps and becomes we can decode its state when we ask. The open gap is that none of these notes tests whether that decoding stays faithful for continuous-thought models like Coconut. The one place it demonstrably works is the filler-token case, where the answer was recoverable from layers the model was actively overwriting.
Sources 9 notes
Multiple architectures—depth-recurrent models, Heima, and Coconut—demonstrate that test-time compute scales through hidden state iteration rather than token generation. This suggests verbalization is a training artifact, not a reasoning requirement.
A 150M-parameter model combining in-context demonstrations with iterative latent computation reached 29.5% pass@2 on ARC-AGI-1 at $0.0007 per task, surpassing previously reported cost-accuracy tradeoffs. The approach separates learning (via demonstrations updating recurrent memory) from reasoning (via iteration in hidden space) without generating intermediate tokens.
Models that re-apply layers in recurrent depth outperform larger feedforward networks on reasoning tasks. This works because recursion enables state tracking and compositional generalization that parameter scaling alone cannot achieve, with convergence signals providing natural halting.
Models trained on systematically irrelevant traces maintain solution accuracy and sometimes improve out-of-distribution generalization, suggesting traces function as computational scaffolding rather than meaningful reasoning steps.
Chain of Draft achieves equivalent accuracy to standard chain-of-thought on arithmetic, symbolic, and commonsense tasks while using only 7.6% of tokens. The 92.4% of removed tokens served style and documentation, not computation.
Show all 9 sources
Logit lens analysis shows models trained with hidden CoT tokens compute correct answers in layers 1-3, then actively suppress these representations in final layers to produce format-compliant filler output. The reasoning is fully recoverable from lower-ranked token predictions.
Five independent mechanisms—RL steering, critique fine-tuning, decoding changes, SAE feature steering, and RLVR—all elicit reasoning already present in base model activations. Post-training selects rather than creates reasoning; the bottleneck is elicitation, not capability acquisition.
Meta's Large Concept Model operates on sentence embeddings rather than tokens, reasoning in a language-agnostic space before decoding to any target language. This hierarchical approach with paragraph-level planning produces more coherent output than flat token generation.
ICE shows that bidirectional attention in diffusion LLMs enables in-place prompting—embedding reasoning directly in masked positions refined alongside answers. Answer confidence converges early while reasoning continues refining, allowing early-exit mechanisms to cut compute by 50% while maintaining accuracy.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Hierarchical Reasoning Model
- Farther the Shift, Sparser the Representation: Analyzing OOD Mechanisms in LLMs
- Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach
- Beyond Semantics: The Unreasonable Effectiveness of Reasonless Intermediate Tokens
- LLM Reasoning Is Latent, Not the Chain of Thought
- Training Large Language Models to Reason in a Continuous Latent Space
- SoftCoT: Soft Chain-of-Thought for Efficient Reasoning with LLMs
- DeepSeek-R1 Thoughtology: Let's think about LLM Reasoning