When an AI slips up midway through a long task, does that one mistake stay small or snowball into more?
Do earlier errors in long tasks increase the likelihood of future mistakes?
This explores whether a mistake made partway through a long task, like a reasoning chain or an agent run, makes later mistakes more likely (snowballing) instead of staying an isolated slip.
This explores whether a mistake made partway through a long task makes later mistakes more likely, or whether it stays a one-off slip. None of the notes retrieved here measure snowballing directly. But several come at it from different sides, and they mostly point to yes: the model's own earlier output becomes what it builds on next.
The most direct mechanism is in Where do memorization errors arise in chain-of-thought reasoning?. It finds that models often predict the next step mostly from the few tokens just before it, and that this local habit accounts for up to 67% of reasoning errors, more so as problems get harder. If a model leans that heavily on what it just wrote, a wrong step is exactly the thing it leans on. Models also make new errors by reacting to themselves. Extended thinking adds self-revision errors (When does thinking too much actually hurt reasoning?), and in untrained models it induces self-doubt that lowers performance (Does extended thinking help or hurt model reasoning?). That second note has a twist: RL training turns the same mechanism into useful gap analysis. So reflecting on earlier steps isn't inherently corrosive. Training decides whether a model treats its earlier steps as evidence to build on or as something to second-guess.
Long tasks are also just long, and length hurts on its own. Reasoning accuracy fell from 92% to 68% with only about 3,000 tokens of padding, well below the context limit (Does reasoning ability actually degrade with longer inputs?). Accuracy also peaks and then declines as thinking gets longer, from 87.3% to 70.3% in one study (Does more thinking time always improve reasoning accuracy?). The best chain length follows an inverted U (Why does chain of thought accuracy eventually decline with length?). So a late-stage failure may come from earlier errors piling up, from length itself, or from both. The corpus doesn't yet separate them.
If errors do compound, catching them early should break the chain, and the evidence fits. Where do reasoning agents actually fail during long traces? argues that most failures in long traces are process violations, not wrong final answers. Checking intermediate states during the run raised task success from 32% to 87%. Scoring only the final answer misses where things went wrong.
The other response is to stop treating mistakes as pure damage. Training on the full messy search, including dead ends and backtracking, produced problem-solvers about 25% more accurate than training on clean optimal paths only (Does training on messy search processes improve reasoning?). Deliberately making errors on few-shot examples and then writing down the lessons also improves reasoning (Does learning from mistakes improve in-context learning?). And agents do better when failed episodes are stored as short abstracted lessons and successes as concrete demonstrations (Should successful and failed episodes be processed differently?). Together these suggest that a mistake mid-task matters less than whether the system can notice it and recover.
Sources 10 notes
STIM framework identifies local, mid-range, and long-range memorization sources in CoT reasoning. Local memorization—based on preceding tokens—accounts for up to 67% of reasoning errors, especially as complexity increases and distributional shift occurs.
Empirical studies demonstrate non-monotonic scaling in test-time reasoning: accuracy peaks at a critical thinking-token count, then declines sharply (87.3% to 70.3% as tokens scale from 1,100 to 16,000). Extended thinking inflates output variance and introduces self-revision errors rather than improving solution quality.
Vanilla models use thinking mode counterproductively, inducing self-doubt that degrades performance. RL training reverses this, transforming the same mechanism into beneficial gap analysis. Training mediates reasoning quality, not just quantity.
FLenQA shows reasoning accuracy drops from 92% to 68% at just 3000 tokens of padding, far below context window capacity. The degradation is task-agnostic, uncorrelated with language modeling performance, and persists even with chain-of-thought prompting.
Increasing thinking tokens from ~1,100 to ~16K reduced benchmark accuracy from 87.3% to 70.3%, revealing a non-monotonic relationship where models overthink easy problems and underthink hard ones.
Show all 10 sources
Task accuracy peaks at intermediate CoT length, with optimal length increasing alongside task difficulty but decreasing with model capability. RL training naturally gravitates toward shorter chains as models improve, revealing that simplicity emerges from reward signals rather than explicit training.
Reliability for long-trace reasoning comes from checking intermediate states and policy compliance during generation, not from scoring final outputs. Adding intermediate verification raised task success from 32% to 87% because most failures are process violations, not wrong answers.
Stream of Search pretraining, which represents exploration and backtracking as serialized strings, achieves 25% higher accuracy than optimal-trajectory-only training. Models learn internal world models for search and adaptive strategies rather than fixed external methods.
LEAP demonstrates that models achieve better performance on reasoning and math tasks by intentionally erring on few-shot examples, reflecting on mistakes, and deriving explicit task-specific principles—without additional labeled data or fine-tuning.
SkillRL demonstrates that treating successful episodes as concrete demonstrations and failures as abstracted lessons achieves state-of-the-art performance on complex tasks while using substantially less context than uniform approaches. The asymmetry mirrors human expert reasoning and avoids the degradation seen in uniform consolidation methods.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Think Deep, Not Just Long: Measuring LLM Reasoning Effort via Deep-Thinking Tokens
- Mining Hidden Thoughts from Texts: Evaluating Continual Pretraining with Synthetic Data for LLM Reasoning
- Does Thinking More always Help? Understanding Test-Time Scaling in Reasoning Models
- When More is Less: Understanding Chain-of-Thought Length in LLMs
- When More Thinking Hurts: Overthinking in LLM Test-Time Compute Scaling
- The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity
- Between Underthinking and Overthinking: An Empirical Study of Reasoning Length and correctness in LLMs
- Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey