If you block one step an AI agent takes, does that stop the harm, or was the danger never in one step?
Can removing a single action prevent a harmful sequence from running?
This explores whether blocking or vetoing one step in an AI agent's chain of actions is enough to stop a harmful outcome, or whether the harm lives in the sequence as a whole.
This explores whether blocking or vetoing one step in an agent's chain of actions is enough to stop a harmful outcome, or whether the harm lives in the sequence as a whole. The corpus leans toward the second. Cutting one action can stop one particular run, but it rarely stops the harmful sequence as a pattern. Research on agent security finds that Can step-by-step approval miss harmful behavior patterns? steps that are each permitted can add up to a broken system constraint. The safety rule applies to the whole run of behavior, so there is no obviously bad step to remove.
Attackers can build for exactly this. Multi-agent systems split work into specialized roles, and that same design lets a harmful goal Can task decomposition hide harmful intent across agents? break into subtasks that each look benign, with the harm appearing only when they combine. The corpus doesn't test what happens when you delete one fragment and watch whether the plan reroutes. It does explain why a step-by-step gate has nothing to hold onto: the intent exists only in the composition.
The gate itself is also weak. Checks that look at one action at a time can't even state a rule that depends on history, because Can stateless checks ever catch sequence-level constraint violations? they see "send file" without knowing it followed "read credentials." Only stateful monitors that track the sequence can. If the gate is a single validator, it becomes a single point of failure. In one four-agent pipeline with authorization turned off, Can a poisoned validator still approve unsafe actions? a validator reading poisoned shared memory approved every unsafe action. And some attacks land before any action exists. Crafted prompts can bias how the workflow is planned, upstream of any step-level inspection (Can prompts alone reshape multi-agent workflows without system access?). Defenses that only inspect the generated workflow Can inspecting generated workflows catch planning-time attacks? arrive too late.
The corpus points to changing what you defend rather than which step you remove. One proposal makes the unit of defense a coordination episode, Should defence units span multiple executions and agents? a set of linked actions whose membership is revised as evidence comes in. Another says containment means controlling what an agent can touch, not just filtering what it says at one moment (Can a model-level filter truly contain an agent with environment access?). One finding is easy to miss. Blocking the final action can make the unsafe-action count read zero, but that number can't tell a contained attack from one that never took hold, Can action-level metrics alone expose contained attacks? so the agent may still be compromised after the veto. A blocked step can look like a fix while the agent stays compromised.
Sources 9 notes
Research shows sequences of individually permissible actions can collectively break system constraints. Safety rules bind entire behavioral envelopes, not just single steps, so checking actions one at a time fails to catch trajectory-level violations.
SafeFlow demonstrates that multi-agent systems' core strength—splitting tasks and specializing roles—creates a safety blind spot where malicious intent can be distributed across steps that each appear benign individually, with harm emerging only in composition.
Per-action checks are structurally unable to state constraints that depend on prior history. Only stateful monitors tracking composed multi-party behavior can verify the behavioral envelopes that prevent individually permissible actions from collectively violating system-level safety.
In a four-agent LangGraph system with authorization disabled, a validator that reads poisoned shared memory forges approvals in every trial, leading to execution of unsafe actions. The validator's position as final authority before action execution, combined with its access to corrupted evidence, made it ineffective as a boundary.
FLOWSTEER demonstrates that a crafted prompt can steer planner-executor systems by biasing workflow formation before infrastructure is invoked, raising malicious success by up to 55 percent. This attack surface exists because contamination enters upstream of workflow inspection defenses.
Show all 9 sources
Defenses that inspect only generated workflows arrive too late to catch FLOWSTEER-style attacks that corrupt planning signals before workflow formation. Input-side defense separating task, methodological, and framing intents reduces malicious success by up to 34 percent by intervening at the instruction-organization boundary.
The operational unit of defence should be a set of actions linked by observed transfers, task authority, and response history, with membership revised as evidence accumulates. Isolated review loses relevant context that spans multiple executions.
A filter judges a single output at one point in time; an agent's risk spreads across memory, retrieved content, tool calls, and environmental reach. Containment requires controlling what an agent can touch, not just what it says now.
The Judgment Bypass Rate measures compromise at the attacked agent rather than at the final action. When the Unsafe Action Rate hits zero across defended conditions, it stops discriminating between attacks that were contained and those that never took hold, making the agent-level reading essential for diagnosis.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- SafeFlow: Semantic Information-Flow Control for Blocking Malicious Propagation in Multi-Agent Systems
- SoK: When Safe Agents Fail Together: The Security of Multi Agent LLM Systems
- Securing Agentic AI: From Per-Action Checks to Trajectory Assurance
- The Troy Moment of AI: Why Some Will Cheat and Some Will Follow?
- EvoSafeHarness: Evolving Model- and Domain-Specific Harnesses for Securing Agents
- PACT: Can Enterprise AI Assistants Be Trusted Under Pressure?
- FLOWSTEER: Prompt-Only Workflow Steering Exposes Planning-Time Vulnerabilities in Multi-Agent LLM Systems
- Counter-Swarm Doctrine: Containing Coordinated Agent Intrusions