Is the structure of reasoning traces learned as a shared stylistic convention?
This explores whether the *shape* of a reasoning trace — its planning, backtracking, step-by-step layout — is a learned formatting habit the model picks up from training, rather than a record of actual computation.
This explores whether the *shape* of a reasoning trace is a learned stylistic convention rather than a window into real computation — and the corpus answers with a fairly emphatic yes. Several notes converge on the idea that what looks like reasoning is mostly a format the model has learned to reproduce. Chain-of-thought is described as constrained imitation of reasoning *form*: models reproduce familiar schemata from training instead of doing novel inference, which is why performance degrades predictably under distribution shift Does chain-of-thought reasoning reveal genuine inference or pattern matching?. The structural cues — format and spatial layout — turn out to matter far more than logical content, with training format shaping reasoning strategy 7.5× more than the actual domain and invalid prompts working about as well as valid ones What makes chain-of-thought reasoning actually work?.
The strongest evidence that structure is convention-not-computation comes from corruption studies: traces deliberately filled with irrelevant or wrong steps teach as well as correct ones, and sometimes generalize *better* out of distribution Do reasoning traces need to be semantically correct?. If semantic correctness can be stripped out and the gains survive, then what's being learned and transmitted is the scaffolding — the look of reasoning — not the reasoning itself. Two notes name this directly: traces are stylistic mimicry, persuasive appearances rather than reliable explanations of computation Do reasoning traces show how models actually think?, and the intermediate tokens of a model like R1 carry no special execution semantics — they're generated identically to any other output and correlate with answers through learned formatting, not function Do reasoning traces actually cause correct answers?.
There's a deeper structural mismatch that reinforces the 'convention' reading. When you actually trace the causal pathways inside the model, the discourse structure the trace *presents* — its tidy linguistic flow of steps — does not match the internal computation; most erroneous steps don't even influence the final answer Do reasoning traces actually show how models think?. So the narrative shape is a surface layer, a shared genre the model writes in, sitting on top of a different (and partly hidden) machine. Trace *length* tells the same story: it tracks how close a problem is to the training distribution rather than how hard the problem genuinely is, decoupling from difficulty entirely out of distribution — length is recall of a learned schema, not adaptive effort Does longer reasoning actually mean harder problems?.
But here's the thing you might not expect: 'stylistic convention' doesn't mean 'inert decoration.' The same corpus shows the structure does real functional work even if it isn't faithful reporting. Certain sentence types — planning and backtracking moves — act as 'thought anchors' that disproportionately steer everything downstream Which sentences actually steer a reasoning trace?, which is precisely what a *learned convention* would look like if the convention had been selected for being useful. And the style is concrete enough to be a measurable, manipulable property: verbose versus concise reasoning occupies distinct regions of activation space, so you can extract a single 'verbosity' vector and dial the style up or down without retraining Can we steer reasoning toward brevity without retraining?. A convention you can locate as a direction in latent space and steer is about as literal a confirmation as you could ask for.
Where does the convention come from, if not from logic? One note points at pretraining: reasoning ability generalizes from broad *procedural* knowledge spread across many documents — the transferable 'how you go about it' patterns — rather than from memorizing specific facts Does procedural knowledge drive reasoning more than factual retrieval?. That's the raw material a shared style is distilled from. The takeaway worth carrying away: a reasoning trace is best read as a *learned genre of writing about thinking* — functionally load-bearing, steerable, and useful, but not a transcript of the computation it appears to describe.
Sources 10 notes
CoT works by constraining models to reproduce familiar reasoning patterns from training, not by enabling novel symbolic reasoning. Performance degrades predictably under distribution shifts—the signature of imitation rather than capability emergence.
Research shows training format shapes reasoning strategy 7.5× more than domain, demo position swings accuracy 20%, and invalid CoT prompts work as well as valid ones. CoT is pattern-guided generation, not formal logic.
Models trained on systematically irrelevant traces maintain solution accuracy and sometimes improve out-of-distribution generalization, suggesting traces function as computational scaffolding rather than meaningful reasoning steps.
LLM reasoning traces perform as persuasive appearances rather than reliable explanations of computation. Invalid logical steps perform nearly as well as valid ones, and corrupted traces generalize comparably, showing that semantic correctness is not what produces the performance gains.
R1's intermediate tokens carry no special execution semantics and are generated identically to other LLM output. Invalid traces frequently produce correct answers, proving traces are not causally necessary—they correlate with answers via learned formatting, not functional reasoning.
ReasoningFlow found that most erroneous steps in traces don't influence final answers, and critically, the discourse structure traces present linguistically does not match their actual internal causal pathways. This gap suggests traces are narrative surface rather than verified computation logs.
Controlled A* maze experiments show trace length correlates with difficulty only in-distribution but decouples entirely out-of-distribution. Trace length primarily reflects recall of training schemas, not adaptive computation.
Counterfactual resampling, attention analysis, and causal suppression all identify planning and backtracking sentences as thought anchors—sparse critical points that guide subsequent reasoning. These are functional pivots, not noise.
Activation-Steered Compression extracts a single vector from 50 paired examples to reduce chain-of-thought length by 67% while maintaining accuracy and achieving 2.73x speedup. The method is training-free and generalizes across model sizes and domains.
Analysis of 5 million pretraining documents shows reasoning relies on broad, transferable procedural knowledge from diverse sources, unlike factual recall which depends on narrow, document-specific memorization of target facts.