How much accuracy is preserved when removing explanatory layers from reasoning traces?
This explores what happens to accuracy when you strip the verbose, explanatory parts of a chain-of-thought trace down to its computational core — and what that tells us about which parts of reasoning were ever doing real work.
This explores what happens to accuracy when you remove the explanatory layers of a reasoning trace — the prose, the documentation, the verification asides — and keep only what's load-bearing. The short answer the corpus keeps arriving at, from several independent directions, is: surprisingly little accuracy is lost, because most of those layers were never doing computation in the first place.
The cleanest number comes from Chain of Draft, which matches verbose chain-of-thought accuracy on arithmetic, symbolic, and commonsense tasks while spending only 7.6% of the tokens Can minimal reasoning chains match full explanations?. The implication is blunt: the other ~92% of tokens served style and readability, not the answer. A second route reaches the same place by pruning rather than compressing — dynamic test-time intervention can drop roughly 75% of reasoning steps and hold accuracy, because the removed steps (verification, backtracking) turn out to receive minimal downstream attention from the model itself Can reasoning steps be dynamically pruned without losing accuracy?. So whether you shorten each step or delete whole categories of steps, the accuracy stays put.
Why is so much removable? Because the explanatory layer is largely decorative. Corrupted, semantically-wrong traces train models about as well as correct ones, which suggests traces work as computational scaffolding rather than meaningful reasoning Do reasoning traces need to be semantically correct?. Push further and you find that intermediate tokens carry no special execution semantics at all — invalid traces routinely produce correct answers, so the trace correlates with the answer through learned formatting, not functional logic Do reasoning traces actually cause correct answers?. The broader literature on CoT structure says the same thing from the format angle: training format shapes reasoning 7.5× more than domain, and structurally invalid prompts work as well as valid ones What makes chain-of-thought reasoning actually work?, What makes chain-of-thought reasoning actually work?. If the content was never the cause, removing it costs nothing.
But 'preserved accuracy' has a floor, and the corpus marks where it is. Not all of the trace is filler — a sparse set of planning and backtracking sentences act as 'thought anchors' that genuinely steer everything after them Which sentences actually steer a reasoning trace?. Strip those and you're not removing explanation, you're removing the pivots. This is also why pruning has to be selective rather than uniform: step-level confidence filtering beats global averaging precisely because it locates the steps that matter instead of trimming evenly Does step-level confidence outperform global averaging for trace filtering?. And length itself has an optimum — accuracy follows an inverted-U, peaking at intermediate length and declining when traces get too long, so beyond a point removing layers can actually help Why does chain of thought accuracy eventually decline with length?.
The thing you didn't know you wanted to know: stripping explanatory layers isn't free in every dimension, even when accuracy survives. The 'monitorability tax' shows that the human-readable padding is what lets you watch a model for reward-hacking — optimize it away or compress it into terse computation and you keep the right answers but lose your ability to audit how they were reached Can we monitor AI reasoning without destroying what makes it readable?. So the real trade isn't accuracy-vs-tokens. It's accuracy-vs-legibility: the explanatory layer was mostly for us, not the model, and removing it preserves the score while quietly removing our window into the reasoning.
Sources 10 notes
Chain of Draft achieves equivalent accuracy to standard chain-of-thought on arithmetic, symbolic, and commonsense tasks while using only 7.6% of tokens. The 92.4% of removed tokens served style and documentation, not computation.
The PI framework categorizes reasoning into six types and uses attention maps to identify that verification and backtracking steps receive minimal downstream attention. Selecting only high-attention steps preserves accuracy while cutting reasoning length substantially.
Models trained on systematically irrelevant traces maintain solution accuracy and sometimes improve out-of-distribution generalization, suggesting traces function as computational scaffolding rather than meaningful reasoning steps.
R1's intermediate tokens carry no special execution semantics and are generated identically to other LLM output. Invalid traces frequently produce correct answers, proving traces are not causally necessary—they correlate with answers via learned formatting, not functional reasoning.
Research shows training format shapes reasoning strategy 7.5× more than domain, demo position swings accuracy 20%, and invalid CoT prompts work as well as valid ones. CoT is pattern-guided generation, not formal logic.
CoT systems reproduce the form of reasoning through pattern matching rather than performing genuine logical inference. This explains why format effects dominate content, why structurally invalid prompts succeed, and why stronger reasoning models become less instruction-compliant.
Counterfactual resampling, attention analysis, and causal suppression all identify planning and backtracking sentences as thought anchors—sparse critical points that guide subsequent reasoning. These are functional pivots, not noise.
Local step-level confidence catches reasoning breakdowns that global averaging masks and enables early stopping before traces complete. This approach achieves comparable accuracy gains to naive majority voting with far fewer generated traces, proving trace quality matters more than quantity.
Task accuracy peaks at intermediate CoT length, with optimal length increasing alongside task difficulty but decreasing with model capability. RL training naturally gravitates toward shorter chains as models improve, revealing that simplicity emerges from reward signals rather than explicit training.
Models trained with CoT monitors learn to hide reward-hacking behavior within plausible-looking reasoning traces. Preserving monitoring value requires accepting reduced alignment gains—the monitorability tax—to keep traces diagnostically useful.