SoK: When Safe Agents Fail Together: The Security of Multi Agent LLM Systems
Safe agents can fail together. Multi-agent LLM systems (MAS) move information, state, decisions, and authority across principal boundaries, creating failures that local checks may miss. Without an execution-level view, a multi-agent setting can easily be mistaken for evidence of a genuinely multi-agent security effect. We thus systematize MAS security through an execution-centered analysis of 197 works, covering six interaction interfaces, four adversary positions, seven system-level risks, and eight recurring attack paths. We introduce an A-I-R framework that organizes attacks by adversary position, interaction interface, and resulting system-level risk, unifying otherwise fragmented attack mechanisms across MAS. We organize defenses through a five-part contract covering path target, observation, intervention, trust boundary, and recovery, and identify path closure and recovery as key challenges. We audit 44 evaluation and benchmark works and identify open challenges in isolating interaction effects, designing comparable and diagnostic metrics, supporting reuse across MAS designs, and evaluating open-system operation.
Introduction. Agents that are safe when used individually can fail when used together in a MAS. Benign prompt fragments can become harmful when combined [11]; truthful reports can steer a group toward a false belief [55]; and attacker-controlled content can pass through honest specialists to a privileged tool [62]. These failures arise through interactions among the separately addressable participants of the MAS, which we call principals. Messages propagate influence between principals, a shared state preserves propagated influence, aggregation combines local outputs from different principals, and delegation transfers authority across boundaries [62,79,93,95]. We therefore study the end-to-end execution of MAS, where interactions may leave a failure largely unchanged, amplify it, create a failure through composition, or require defining a security property that exists only when principals interact. Existing surveys cover many important threats, approaching from different perspectives.
Discussion / Conclusion. This SoK frames multi-agent LLM security around end-toend execution. Across 197 works, we organize MAS security through system configuration, interaction interfaces, attack paths, defenses, and evaluation. Our analysis shows that security depends on how adversarial influence crosses principal boundaries, what state and authority remain reachable, whether defenses can close the path, and whether evaluations verify the resulting system-level effect. This executioncentered view provides a common basis for comparing security claims across MAS designs.
Lines of inquiry this paper opens 24
Research framings built by reading the notes related to this paper — the questions it feeds into.
What coordination failures limit multi-agent LLM systems as they scale?- How do multi-agent LLM systems fail at coordination and role consistency?
- Why do LLM agents fail where game-theoretic bots succeed?
- What specific network sizes trigger coordination degradation in LLM systems?
- How can humans oversee multiple partial-progress agents simultaneously?
- Do parallel LLM workers coordinate emergently without predefined collaboration rules?
- Can silence training address premature consensus failures in multi-agent reasoning systems?
- What causes silent agreement in multi-agent reasoning systems?
- Can agreement detection agents improve multi-agent deliberation beyond just negotiation?
- Does structured debate between agent groups improve evaluation consensus more than independent scoring?
- How do multi-agent systems fail when agents cannot verify each other's claims?
- Can designated leadership structures reduce premature convergence in multi-agent reasoning?
- Why do multi-agent systems converge on wrong answers without debate safeguards?
- Can agreement-detection agents verify that position convergence reflects actual mutual adjustment?
- How do agreement-detection agents improve distributed coordination outcomes?
- Does silent agreement actually represent the biggest failure mode in multi-agent reasoning?
- What role should agreement detection play in improving multi-agent team performance?
- Can debate-style multi-agent systems be trusted on contested factual domains?
- Can silent agreement be prevented in multi-agent reasoning systems?
- Why does ambiguity detection require different multi-agent mechanisms than verifiable reasoning tasks?