SYNTHESIS NOTE
Topics›Agents Multi Architecture›this note

Can individually safe agents fail when working together?

When multiple AI agents interact—sharing information, state, and authority—do failures emerge that local safety checks alone cannot catch? This matters because system-level safety depends on understanding how principals interact.

Synthesis note · 2026-09-23 · sourced from Agents Multi Architecture

The paper's opening sentence is also its argument: "Agents that are safe when used individually can fail when used together in a MAS." Its introduction gives three shapes of the failure, each cited by number only. "Benign prompt fragments can become harmful when combined", "truthful reports can steer a group toward a false belief", and "attacker-controlled content can pass through honest specialists to a privileged tool". In none of the three is any single participant the culprit. The paper calls the separately addressable participants principals, and locates the failure in what happens between them: "Messages propagate influence between principals, a shared state preserves propagated influence, aggregation combines local outputs from different principals, and delegation transfers authority across boundaries" (see How do failures cross boundaries between multiple agents?).

The vault holds each of the three shapes in a different guise. The first is the fragmentation case in Can task decomposition hide harmful intent across agents?. The third resembles the injection-through-a-relay surface in Do internal agent hops in pipelines need security monitoring?, and it is closer still to Can forwarded content trick high-privilege agents into misusing their authority?, which puts nearly the same route under the classic name and reports it measured in one pipeline. That matching is my reading, since the excerpt does not say which prior works it cites. The second has no counterpart yet, which is why it is filed as Can true reports together mislead a group?. What the SoK adds to Can individual components pass safety checks if the system still fails? is a claim of scale: 197 works, organized around end-to-end execution rather than around any one attack.

The conclusion turns the thesis into four questions to put to any multi-agent security claim: how adversarial influence crosses principal boundaries, what state and authority remain reachable, whether defenses can close the path, and whether evaluations verify the resulting system-level effect. The paper offers this as "a common basis for comparing security claims across MAS designs."

The strongest objection is that this is true of any composed system, since the whole is more than its parts. The paper's reply is in its abstract: it separates interactions that change a failure from those that leave it alone (see Does a multi-agent setting automatically signal a security effect?).

What the excerpt does not give. No per-example detail, no results, no characterization of the 197-work corpus, and no identification of the three cited works. It is a framing and an organizing claim, not a measurement.

Inquiring lines that read this note 12

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

How can workflow-level validation detect semantic corruption that protocol compliance misses? How do coordinated agent sequences violate constraints that individual actions respect? Where do unmonitored channels leave multi-agent planning vulnerable to attack? What determines whether AI system errors remain visible and contestable? What coordination and communication failures emerge in multi-agent LLM systems? Why does single-turn training fail to generalize to multi-turn tasks? How can multi-agent LLM systems maintain genuine reasoning diversity without premature convergence?

Related concepts in this collection 4

This note in its neighbourhood — explore the map, then jump to a related concept in the list below.

Concept map
16 direct connections · 94 in 2-hop network ·medium cluster Open in graph ↗

Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph

your link semantically near linked from elsewhere

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Original note title

safe agents can fail together — multi-agent failures arise through interactions among principals that local checks may miss