If you wall off a hacked AI agent's memory, is the system actually healed, or just kept from getting worse?
Does quarantining state count as recovery in multi-agent attack scenarios?
This explores whether walling off a compromised agent's memory or shared state actually restores a multi-agent system, or only stops the damage from spreading.
This explores whether walling off a compromised agent's memory or shared state actually restores a multi-agent system, or only stops the damage from spreading. The corpus has no note that tests quarantine-as-recovery directly. The neighboring findings point one way: quarantine is containment, and it counts as recovery only when the failure lived entirely in the state you isolated.
The trouble is that state is only one of four routes a failure can take. How do failures cross boundaries between multiple agents? lists them: messages carry influence, shared state preserves it over time, aggregation combines corrupted outputs, and delegation transfers authority. Quarantining state addresses the second verb, and by the time you act, the influence may already have left through the other three. Can one compromised agent corrupt an entire multi-agent network? shows how bad that can be. One biased agent passed persistent behavioral corruption through six downstream agents using ordinary messages, and the bias slipped past detection and paraphrasing defenses because it carried no explicit meaning. Isolate the source and the six downstream agents still carry the bias, with nothing readable to point at.
Some attacks leave nothing to quarantine. Can task decomposition hide harmful intent across agents? describes harmful goals split into steps that each look benign, with harm appearing only when they combine. Can agents be tricked into delegating work in circles? sends tasks around in circles through delegation, an attack with no single-agent counterpart. Can prompts alone reshape multi-agent workflows without system access? shows a crafted prompt shaping how the workflow is formed, upstream of the defenses that inspect it, raising malicious success by up to 55 percent. In each case the fault sits in how agents combine, delegate, or get planned, not in a memory store you can lock away. Does a multi-agent setting automatically signal a security effect? gives the sorting rule. Only amplification, composition and emergent effects are truly multi-agent, and those are the ones a per-agent quarantine misses.
Quarantine does earn its place as containment. Can a model-level filter truly contain an agent with environment access? argues that risk spreads across memory, retrieved content, tool calls and environmental reach, so control means limiting what an agent can touch rather than judging what it says. Can explicit authorization boundaries prevent agents from modifying protected tests? adds a warning about how to do it. Boundaries kept protected tests intact only when paired with restricted tools and when they named the protected state itself. A stated prohibition alone wasn't enough, so a quarantine that is declared but not enforced by capability isn't one.
The closest the corpus comes to a recovery answer is How can operators stop coordinated agent intrusions now?. Its defense preserves relationships across executions and ties responses to persistent state rather than closed channels. That reads as recovery meaning you track an intrusion through time and across agents, not reset one node. A practical test is to ask which of the four mechanisms carried the failure. If it was only shared state, quarantine plausibly recovers the system. If it was anything else, quarantine has bought you time.
Sources 9 notes
Research identifies four verbs describing how failures propagate in multi-agent systems: messages propagate influence between principals, shared state preserves it over time, aggregation combines potentially corrupted local outputs, and delegation transfers authority across boundaries. Each mechanism operates independently of pipeline topology.
Research demonstrates that a single biased agent can transmit persistent behavioral corruption through six downstream agents in chain and bidirectional topologies using only normal inter-agent communication. The bias evades detection and paraphrasing defenses because it carries no explicit semantic content.
SafeFlow demonstrates that multi-agent systems' core strength—splitting tasks and specializing roles—creates a safety blind spot where malicious intent can be distributed across steps that each appear benign individually, with harm emerging only in composition.
Research identifies a novel MAS-specific attack that weaponizes cross-agent delegation to form task cycles, distinct from applying existing attacks like prompt injection to teams. The mechanism requires multiple agents and has no single-agent counterpart.
FLOWSTEER demonstrates that a crafted prompt can steer planner-executor systems by biasing workflow formation before infrastructure is invoked, raising malicious success by up to 55 percent. This attack surface exists because contamination enters upstream of workflow inspection defenses.
Show all 9 sources
Interaction between agents can leave failures unchanged, amplify them, create them through composition, or define new properties. Only amplification, composition, and emergent properties qualify as genuinely multi-agent effects; unchanged failures reflect single-agent problems repackaged.
A filter judges a single output at one point in time; an agent's risk spreads across memory, retrieved content, tool calls, and environmental reach. Containment requires controlling what an agent can touch, not just what it says now.
Testing showed that explicit authorization boundaries kept protected tests unmodified only when paired with restricted tools. Naming a prohibition was insufficient; boundaries must specify the protected state itself to be effective.
The doctrine preserves relationships across executions, constrains shared resources agents can access, and ties responses to persistent state rather than closed channels. Operators can implement this through collaboration policy and permission-level testing now.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- SoK: When Safe Agents Fail Together: The Security of Multi Agent LLM Systems
- Agents of Chaos
- SafeFlow: Semantic Information-Flow Control for Blocking Malicious Propagation in Multi-Agent Systems
- Securing Agentic AI: From Per-Action Checks to Trajectory Assurance
- FLOWSTEER: Prompt-Only Workflow Steering Exposes Planning-Time Vulnerabilities in Multi-Agent LLM Systems
- Emergent Collusion in Long-Horizon LLM Agent Interaction
- Counter-Swarm Doctrine: Containing Coordinated Agent Intrusions
- Can AI Agents Agree?