Why does intermediate step quality predict reasoning outcomes better than global features?
This explores why local, step-by-step signals inside a reasoning trace turn out to be sharper predictors of whether the answer comes out right than coarse, whole-trace measures like overall confidence or total length.
This explores why local, step-by-step signals inside a reasoning trace turn out to be sharper predictors of whether the answer comes out right than coarse, whole-trace measures like overall confidence or total length. The corpus points to a single underlying reason: reasoning succeeds or fails at specific moments, and averaging over the whole chain washes those moments out.
The most direct evidence is that step-level confidence beats global confidence averaging when filtering traces — local scoring catches a reasoning breakdown at the exact step it happens, while a global average lets one fatal misstep hide behind many fluent-looking ones Does step-level confidence outperform global averaging for trace filtering?. The same logic shows up in where the learning signal actually lives: only about 20% of tokens are high-entropy 'forking points' where the model chooses a direction, and training on just those matches full-gradient updates Do high-entropy tokens drive reasoning model improvements?. Most of a trace is low-stakes continuation; the outcome is decided at a few junctions. A global feature can't see junctions — it only sees the blur.
This is also why judges that reason about individual steps outperform classifiers that score a trace as one lump Can judges that reason about reasoning outperform classifier rewards?, and why the failure modes that most damage reasoning are themselves local events: underthinking is premature switching away from a path at a particular step, and penalizing those transition moments improves accuracy with no retraining at all Do reasoning models switch between ideas too frequently?. The diagnostic resolution that matters is the step, not the chain.
Here's the twist that makes this more than obvious bookkeeping. Global features like total length are not just weak predictors — they can be actively misleading. Optimal chain length follows an inverted-U, so 'longer' tells you almost nothing about quality on its own Why does chain of thought accuracy eventually decline with length?, and minimal chains match verbose ones at a fraction of the tokens because most of the words were style, not computation Can minimal reasoning chains match full explanations?. The signal was never in the volume; it was in a few load-bearing steps.
And then the genuinely unsettling note: traces deliberately corrupted with irrelevant content can teach as well as correct ones, which suggests the steps sometimes work as computational scaffolding rather than meaningful logic Do reasoning traces need to be semantically correct?. Put next to the filtering result, that's a real tension worth sitting with — 'step quality' that predicts outcomes may be measuring how well a step structures computation, not whether it's semantically true. The step is the right unit of measurement; what the measurement actually captures is still being argued out.
Sources 7 notes
Local step-level confidence catches reasoning breakdowns that global averaging masks and enables early stopping before traces complete. This approach achieves comparable accuracy gains to naive majority voting with far fewer generated traces, proving trace quality matters more than quantity.
Only ~20% of tokens exhibit high entropy as pivotal reasoning decision points; RLVR primarily adjusts these forking tokens. Training exclusively on them matches or exceeds full-gradient performance, revealing that the minority carries the learning signal.
StepWiser demonstrates that training judges to produce reasoning chains about policy reasoning—rather than classify steps—yields better judgment accuracy and data efficiency. Independent confirmation from GenPRM and ThinkPRM shows generative PRMs outperform discriminative ones with orders of magnitude less training data.
o1-like models frequently abandon reasoning paths mid-exploration, wasting tokens on incomplete approaches. A decoding-only penalty on thought-transition tokens (TIP strategy) discourages switching, improving accuracy on challenging math without model fine-tuning.
Task accuracy peaks at intermediate CoT length, with optimal length increasing alongside task difficulty but decreasing with model capability. RL training naturally gravitates toward shorter chains as models improve, revealing that simplicity emerges from reward signals rather than explicit training.
Chain of Draft achieves equivalent accuracy to standard chain-of-thought on arithmetic, symbolic, and commonsense tasks while using only 7.6% of tokens. The 92.4% of removed tokens served style and documentation, not computation.
Models trained on systematically irrelevant traces maintain solution accuracy and sometimes improve out-of-distribution generalization, suggesting traces function as computational scaffolding rather than meaningful reasoning steps.