Small mistakes at each step of a handed-off job look harmless alone, so how do they quietly add up to real damage?
Why do sparse per-step errors accumulate undetected across delegated tasks?
This explores why tiny, occasional mistakes at individual steps of a long, handed-off task pile up into serious damage without anyone noticing, and what the corpus says about catching them.
This explores why tiny, occasional mistakes at individual steps of a long, handed-off task pile up into serious damage without anyone noticing. The corpus suggests the errors stay hidden because every place we normally look is either too local or too global to see them.
The clearest evidence is a study of long relay workflows, where a document is passed from step to step. Even the strongest frontier models degraded about 25% of the document's content, and the damage decelerated but never plateaued. Spot-checked outputs still looked fine (Do frontier LLMs silently corrupt documents in long workflows?). No single step fails visibly. Each one changes only a little, so any one output passes inspection. The harm exists only in the running total, and nobody inspects that.
Three other notes in the corpus describe the same blind spot from different angles. Per-action guardrails are structurally unable to state a rule that depends on history, so a series of individually permissible actions can add up to a violation no single check flags (Can stateless checks ever catch sequence-level constraint violations?). Scoring only the final answer misses most failures, because most are process violations rather than wrong answers. Adding intermediate verification raised task success from 32% to 87% (Where do reasoning agents actually fail during long traces?). Averaging confidence over a whole reasoning trace also dilutes the signal: a local breakdown gets washed out, while step-level confidence catches it (Does step-level confidence outperform global averaging for trace filtering?). A sparse error is invisible up close (each step looks fine) and invisible from far away (the total looks plausible).
The fixes all put a check at every step rather than only at the end. One approach makes verification run asynchronously alongside generation. It steps in only when it finds a violation, and on correct runs it adds almost no delay (Can verifiers monitor reasoning without slowing generation down?). Another goes to the extreme, splitting a task into minimal subtasks with voting on each step and flagging correlated errors. This ran a million-step task with zero errors, and small non-reasoning models were enough (Can extreme task decomposition enable reliable execution at million-step scale?). A milder version wraps the model in an explicit algorithm that gives each call only the context its step needs, so each step is a small, debuggable unit (Can algorithms control LLM reasoning better than LLMs alone?).
None of these notes explains why models introduce sparse errors in the first place. The corpus does show that once the errors exist, delegation hides them, and that the remedy is to check at every step instead of trusting the final result.
Sources 7 notes
Even the strongest models (Gemini 3.1 Pro, Claude 4.6 Opus, GPT 5.4) degrade documents by ~25% over long relay workflows across 52 domains. Degradation decelerates but never plateaus, and errors compound silently, remaining undetected in spot-checked outputs.
Per-action checks are structurally unable to state constraints that depend on prior history. Only stateful monitors tracking composed multi-party behavior can verify the behavioral envelopes that prevent individually permissible actions from collectively violating system-level safety.
Reliability for long-trace reasoning comes from checking intermediate states and policy compliance during generation, not from scoring final outputs. Adding intermediate verification raised task success from 32% to 87% because most failures are process violations, not wrong answers.
Local step-level confidence catches reasoning breakdowns that global averaging masks and enables early stopping before traces complete. This approach achieves comparable accuracy gains to naive majority voting with far fewer generated traces, proving trace quality matters more than quantity.
Decoupling verification from generation lets verifiers run alongside a single trace, forking to extract verifiable state and intervening only on violations. On correct runs the latency penalty is near-zero; interwhen matches or beats CoT across benchmarks at similar token budgets.
Show all 7 sources
MAKER solves million-step tasks with zero errors by decomposing into minimal subtasks, applying voting at each step, and flagging correlated errors. Surprisingly, small non-reasoning models suffice when decomposition is extreme enough, inverting the standard approach to hard problems.
LLM Programs embed LLMs within explicit algorithms that manage control flow and state, presenting only step-specific context to each LLM call. This information hiding addresses capability and context window limits while treating complex reasoning as modular, debuggable sub-tasks.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Local Coherence or Global Validity? Investigating RLVR Traces in Math Domains
- interwhen: A Generalizable Framework for Steering Reasoning Models with Test-time Verification
- Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces!
- What Characterizes Effective Reasoning? Revisiting Length, Review, and Structure of CoT
- Diagnosing Harmful Continuation in Answer-Correct Long-CoT Training Traces
- Beyond Semantics: The Unreasonable Effectiveness of Reasonless Intermediate Tokens
- Rethinking Thinking Tokens: LLMs as Improvement Operators
- The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMs