Can articulating latent reasoning processes improve transfer across domains?
This explores whether making hidden, unspoken reasoning steps explicit — spelling out the verification, recall, and self-talk that experts skip on the page — actually helps a model carry skills from one domain into another, rather than just memorizing the training domain.
This explores whether making hidden reasoning explicit improves transfer across domains — and the corpus's most direct answer is a qualified yes, with an important twist about *why* it works. The cleanest evidence comes from work showing that expert texts are only the surface residue of thinking: the writer's verification, knowledge recall, and self-talk got deleted before the words hit the page. When training data is augmented to reconstruct those missing thought processes, the resulting reasoning skills transfer across domains and even adapt their depth to how hard a problem is, beating standard continual pretraining by up to 8 points on the hardest problems Can reconstructing expert thinking improve reasoning transfer?. So articulation isn't just decoration — it supplies the connective procedure that makes a skill portable.
Why would spelling out *process* travel better than spelling out *answers*? Because reasoning and recall live in different places. An analysis of five million pretraining documents found that reasoning leans on broad, transferable procedural knowledge drawn from many sources, while factual recall depends on narrow, document-specific memorization Does procedural knowledge drive reasoning more than factual retrieval?. There's even an architectural echo of this: knowledge retrieval seems to sit in lower network layers and reasoning adjustment in higher ones, which is why training a model hard on reasoning can lift math while quietly degrading knowledge-heavy fields like medicine Why does reasoning training help math but hurt medical tasks?. Articulating *procedure* targets the transferable part; piling on domain facts does not.
The twist is unsettling. A second strand of the corpus suggests the *content* of the articulated reasoning may matter far less than the fact that articulation is happening at all. Models trained on deliberately corrupted, irrelevant reasoning traces stay just as accurate — and sometimes generalize better out of distribution — which implies the traces work as computational scaffolding rather than as meaningful logic Do reasoning traces need to be semantically correct?. That pairs with the sobering finding that chain-of-thought reasoning is distribution-bound: shift the task, length, or format and models keep producing fluent reasoning that is logically hollow Does chain-of-thought reasoning actually generalize beyond training data?. So explicit reasoning can *look* like it transfers while really just imitating the form of reasoning.
There's also a deeper question of whether reasoning needs to be articulated in words at all. Base models appear to already contain latent reasoning that minimal training merely unlocks — post-training selects reasoning rather than creating it Do base models already contain hidden reasoning ability?. Some approaches skip language entirely: looped pretraining performs iterative reasoning in latent space and produces traces that are arguably *more* honest than spoken chain-of-thought Can reasoning happen in latent space during pretraining?, while large concept models reason over sentence-level embeddings in a language-agnostic space before decoding Can reasoning happen at the sentence level instead of tokens?. Quiet-STaR splits the difference, training a model to generate a rationale at every token of ordinary text so general reasoning emerges as a side effect of better prediction Can models learn reasoning from predicting any text?.
The synthesis a reader might not expect: articulation helps transfer mainly when it surfaces the *procedure* an expert hid, not the answer they reached — and even then, part of the benefit may come from the act of unrolling computation rather than from the reasoning being correct. The most reliable gains come from rewarding the *quality* of explanation, as when reinforcement learning cycles between augmented and bare generation to internalize coherent knowledge structures better than supervised fine-tuning on correct tokens alone Can reinforcement learning embed domain knowledge more effectively than supervised fine-tuning?. Articulating latent reasoning is a real lever for cross-domain transfer — just not for the tidy reason it first appears to be.
Sources 10 notes
Training on expert texts augmented with reconstructed thought processes (self-talk, knowledge recall, verification) produces reasoning skills that transfer across domains and adapt depth to problem difficulty, outperforming standard continual pretraining by up to 8 points on hard problems.
Analysis of 5 million pretraining documents shows reasoning relies on broad, transferable procedural knowledge from diverse sources, unlike factual recall which depends on narrow, document-specific memorization of target facts.
Two-phase inference model shows knowledge retrieval operates in lower network layers while reasoning adjustment happens in higher layers. This separation explains why reasoning training improves math but can degrade knowledge-intensive domains like medicine.
Models trained on systematically irrelevant traces maintain solution accuracy and sometimes improve out-of-distribution generalization, suggesting traces function as computational scaffolding rather than meaningful reasoning steps.
DataAlchemy experiments show CoT fails systematically under distributional shifts in task, length, and format. Models produce fluent but logically inconsistent reasoning — imitating reasoning form without valid underlying logic.
Five independent mechanisms—RL steering, critique fine-tuning, decoding changes, SAE feature steering, and RLVR—all elicit reasoning already present in base model activations. Post-training selects rather than creates reasoning; the bottleneck is elicitation, not capability acquisition.
Ouro models achieve 2–3× efficiency gains by performing iterative reasoning in latent space during pretraining, not through extra capacity. Their intermediate predictions align faithfully with final outputs, making latent traces more honest than explicit chain-of-thought reasoning.
Meta's Large Concept Model operates on sentence embeddings rather than tokens, reasoning in a language-agnostic space before decoding to any target language. This hierarchical approach with paragraph-level planning produces more coherent output than flat token generation.
Quiet-STaR trains language models to generate rationales at every token position during pretraining on arbitrary internet text, enabling general reasoning without task-specific datasets. Rationale quality is judged by predictive accuracy rather than labeled correctness, allowing reasoning competence to emerge as a side effect of improved language modeling.
RLAG rewards both answer accuracy and explanation rationality by cycling between augmented and unaugmented generation, progressively internalizing coherent knowledge structures. This outperforms SFT because it prioritizes reasoning quality over token-level correctness.