Can individually safe agents fail when working together?
When multiple AI agents interact—sharing information, state, and authority—do failures emerge that local safety checks alone cannot catch? This matters because system-level safety depends on understanding how principals interact.
The paper's opening sentence is also its argument: "Agents that are safe when used individually can fail when used together in a MAS." Its introduction gives three shapes of the failure, each cited by number only. "Benign prompt fragments can become harmful when combined", "truthful reports can steer a group toward a false belief", and "attacker-controlled content can pass through honest specialists to a privileged tool". In none of the three is any single participant the culprit. The paper calls the separately addressable participants principals, and locates the failure in what happens between them: "Messages propagate influence between principals, a shared state preserves propagated influence, aggregation combines local outputs from different principals, and delegation transfers authority across boundaries" (see How do failures cross boundaries between multiple agents?).
The vault holds each of the three shapes in a different guise. The first is the fragmentation case in Can task decomposition hide harmful intent across agents?. The third resembles the injection-through-a-relay surface in Do internal agent hops in pipelines need security monitoring?, and it is closer still to Can forwarded content trick high-privilege agents into misusing their authority?, which puts nearly the same route under the classic name and reports it measured in one pipeline. That matching is my reading, since the excerpt does not say which prior works it cites. The second has no counterpart yet, which is why it is filed as Can true reports together mislead a group?. What the SoK adds to Can individual components pass safety checks if the system still fails? is a claim of scale: 197 works, organized around end-to-end execution rather than around any one attack.
The conclusion turns the thesis into four questions to put to any multi-agent security claim: how adversarial influence crosses principal boundaries, what state and authority remain reachable, whether defenses can close the path, and whether evaluations verify the resulting system-level effect. The paper offers this as "a common basis for comparing security claims across MAS designs."
The strongest objection is that this is true of any composed system, since the whole is more than its parts. The paper's reply is in its abstract: it separates interactions that change a failure from those that leave it alone (see Does a multi-agent setting automatically signal a security effect?).
What the excerpt does not give. No per-example detail, no results, no characterization of the 197-work corpus, and no identification of the three cited works. It is a framing and an organizing claim, not a measurement.
Inquiring lines that read this note 12
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
How can workflow-level validation detect semantic corruption that protocol compliance misses? How do coordinated agent sequences violate constraints that individual actions respect?- Why do individual safe actions create unsafe behavior collectively?
- What makes an advisory instruction fail when a task is split across agents?
- What happens when stopping rules must cross organizational boundaries?
- Why are unmonitored channels between agents a safety risk?
- Why does monitoring performed by agents on agents create safety risks?
Related concepts in this collection 4
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Can individual components pass safety checks if the system still fails?
Explores whether local validation at each step—alignment checks, protocol compliance, plausibility tests—can guarantee safety when components are composed into larger workflows. Why the gap between component-level assurance and system-level outcomes matters for AI safety.
extends: the same shape, stated at survey scale with the principal-boundary vocabulary
-
Can forwarded content trick high-privilege agents into misusing their authority?
When low-privilege agents retrieve and forward information to higher-privilege agents, does the content itself create conditions where the privileged agent's legitimate authority gets misdirected? This matters because role separation in multi-agent systems assumes the hierarchy protects against misuse.
the third shape under its classic name, with a measured undefended pipeline behind it; the match is the vault's, since the SoK cites its three cases by number only
-
Who actually bears the risk when multi-agent workflows fail?
When AI agents delegate tasks across organizations, the people harmed by failures may never see the workflow or author the prompts. This explores whether current oversight designs protect the right parties.
the human side of principals: who bears a failure that no single principal produced
-
How do agent security layers connect across the stack?
Agent security is often treated as separate challenges at each layer—inputs, delegation, routing, containment. But do defenses at one layer fail if others aren't secured? This explores whether securing agents requires end-to-end integration.
a parallel survey-level thesis cut by scope rather than by interaction
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- SoK: When Safe Agents Fail Together: The Security of Multi Agent LLM Systems
- EvoSafeHarness: Evolving Model- and Domain-Specific Harnesses for Securing Agents
- Securing Agentic AI: From Per-Action Checks to Trajectory Assurance
- The safety failures we are not instrumenting: a perspective on hidden safety-critical challenges in modern AI systems
- Agents of Chaos
- The Troy Moment of AI: Why Some Will Cheat and Some Will Follow?
- LLMs Corrupt Your Documents When You Delegate
- Unaccountable Delegation, Fading Skills: Mapping the Risks of Workplace AI Agents
Original note title
safe agents can fail together — multi-agent failures arise through interactions among principals that local checks may miss