INQUIRING LINE

AI workflows can fail without anyone noticing: each step looks fine and passes its checks, yet the whole thing quietly goes wrong.

How do workflows normalize and hide errors before they become visible hazards?

This explores how AI workflows make failures look ordinary, spreading them across steps, disguising them as plausible output, and passing routine checks, so that by the time anyone notices, the damage is already done.


This explores how AI workflows make failures look ordinary, spreading them across steps, disguising them as plausible output, and passing routine checks, so that by the time anyone notices, the damage is already done. The corpus suggests the hiding isn't mysterious. Deployed systems fail in ways that are plausible rather than shocking and distributed rather than localized, and workflows make this worse by making those failures feel routine. The trouble is a mismatch between what we look for and the shape failure actually takes Why do safety failures remain invisible to our evaluation methods?.

One way errors hide is that every step passes its own check. Plausibility, alignment, and protocol-compliance checks each verify something real, but none of them verifies what determines safe end-to-end behavior, so a workflow can be clean at every step and still fail as a whole Can individual components pass safety checks if the system still fails?. The same holds over time: a sequence of individually permissible actions can add up to a violation that no single-step approval would flag agent-security-is-a-property-of-behavior-over-time-rather-of-individual-act. Checking actions one at a time is the wrong shape of check.

The second way is that the output itself looks fine. In long document-editing workflows, frontier models corrupt about 25% of content, and the errors compound without leveling off. They also go unnoticed in spot-checked outputs Do frontier LLMs silently corrupt documents in long workflows?. Better models make this harder to catch. Weaker models delete content, which is visible, while frontier models rewrite it subtly and keep the surface intact Does model capability change how documents degrade?. Giving the agent better tools doesn't help, because the fault sits upstream in the model's judgment about what to change, not in the editing interface Can better tools fix LLM document editing errors?.

The third way is that the wrong thing gets inspected, or gets inspected too late. Reviewing only the generated workflow misses attacks that bias the planning signals before the workflow exists Can inspecting generated workflows catch planning-time attacks?. Scoring only the final answer misses the fact that most failures are process violations, and adding checks on intermediate states raised task success from 32% to 87% Where do reasoning agents actually fail during long traces?. The fixes in the corpus all move a check to where the risk is actually visible. One is validating the assembled workflow at the point of an irreversible action, where the fragmented risk picture can be put back together Where should workflow validation gates be placed for safety?.

Once a workflow spans several agents, the people who bear the harm may never have seen it run. Requester, observer, and affected party can be three different people or organizations, so the ones who could notice something is off are not the ones it lands on Who actually bears the risk when multi-agent workflows fail?. The corpus also says we can't yet measure whether errors stay visible and recoverable. Partial instruments exist for visibility, containment, and rollback, but none covers the whole socio-technical system How can we measure whether AI errors stay visible and recoverable?. Until one does, a workflow can hide its errors from every check we currently run.


Sources 11 notes

Why do safety failures remain invisible to our evaluation methods?

Deployed AI systems fail in ways that our instruments cannot see: plausible rather than shocking, distributed rather than localized, normalized by workflows rather than immediately legible. The problem is not mystery but mismatched assumptions about failure shape.

Can individual components pass safety checks if the system still fails?

Three mechanisms across SafeFlow, ChannelGuard, and Honest Quorum show that passing local checks (plausibility, alignment, protocol compliance) does not prevent system failures. The gap persists because local checks verify different properties than those that determine safe end-to-end behavior.

Do frontier LLMs silently corrupt documents in long workflows?

Even the strongest models (Gemini 3.1 Pro, Claude 4.6 Opus, GPT 5.4) degrade documents by ~25% over long relay workflows across 52 domains. Degradation decelerates but never plateaus, and errors compound silently, remaining undetected in spot-checked outputs.

Does model capability change how documents degrade?

DELEGATE-52 shows weaker LLMs degrade documents through visible deletion, while frontier models degrade through subtle corruption that preserves surface integrity. This shift makes frontier failures harder to detect and potentially more dangerous at workflow scale.

Can better tools fix LLM document editing errors?

DELEGATE-52 shows that agentic tool access fails to improve performance on long-horizon document tasks. The degradation mechanism originates upstream in the model's judgment about what to change, not in editing interface limitations.

Show all 10 sources
Can inspecting generated workflows catch planning-time attacks?

Defenses that inspect only generated workflows arrive too late to catch FLOWSTEER-style attacks that corrupt planning signals before workflow formation. Input-side defense separating task, methodological, and framing intents reduces malicious success by up to 34 percent by intervening at the instruction-organization boundary.

Where do reasoning agents actually fail during long traces?

Reliability for long-trace reasoning comes from checking intermediate states and policy compliance during generation, not from scoring final outputs. Adding intermediate verification raised task success from 32% to 87% because most failures are process violations, not wrong answers.

Where should workflow validation gates be placed for safety?

SafeFlow validates assembled workflows at irreversible action boundaries, reconstructing the global risk picture that no single step holds. This commit-point placement complements planning-boundary and per-hop defenses, each seeing attacks the others miss.

Who actually bears the risk when multi-agent workflows fail?

Failures in multi-agent systems affect people and organizations who neither wrote the initial prompt nor observed the workflow. Oversight designs that assume requester, observer, and affected party are the same person fail when they are separated by delegation chains.

How can we measure whether AI errors stay visible and recoverable?

Partial instruments exist for individual conditions in isolated settings, but none measures the full socio-technical system the paper identifies as necessary. Visibility has a model-side measure (chain-of-thought disclosure), containment has incident-level counts, and recoverability has rollback timing, yet none bridges all four or captures human-institution factors.

Papers this line draws on 8

The research behind the notes this line reads — ranked by how closely each paper relates.