Where do safety wins come from in multi-agent systems?

ChannelGuard's headline finding is about measurement before it is about defense. Across 2,100 traces, eight attack families, five defenses and three model backends, an undefended pipeline scored attack success 0.000 on tool- and memory-poisoning. A standard report would call that fully safe. The paper then traces where the blocks came from: on Azure GPT-5, 54 of 60 were the cloud provider's server-side filter. Run the same pipeline on a backend without such a filter and the safety is silently re-sourced to the agent model's own alignment. Outcome reporting records that attacks failed. It does not record which layer stopped them.

Three things follow. First, a favorable score is ambiguous in the same way a failing score is. What causes failures in exploitation benchmarks? argues that a low exploit score is a lower bound on capability because non-capability causes also produce failures. This is the mirror image: a zero attack-success score has a cause that is not the application's own defense. Second, the layer doing the protecting is one the builder cannot inspect. The paper describes provider-side filters as opaque and stochastic, so an application that leans on one has inherited a safety property it can neither audit nor hold constant across a model swap. Third, the dependence is invisible until the backend changes, and then only if someone attributes the blocks.

This extends Does a single benchmark score actually predict agent readiness? from capability to safety: one outcome column hides the layers behind it. It also gives Do internal agent hops in pipelines need security monitoring? its stakes, because an application with an open interior and a provider filter at the perimeter looks safe exactly as long as the filter holds.

The vault now holds other outcome numbers that cannot say what produced them, each with its own hidden cause. Can action-level metrics alone expose contained attacks? finds an action-level 0 percent that reads the same whether an attack never took hold or was contained. How many GPT-MAS failures came from tool access confusion? finds failures that were an agent's wrong belief about its tools and not the defense. What behaviors hide behind a zero crossing rate? finds one zero covering four different stopping policies. Can a correct outcome hide protocol violations in multi-agent systems? finds a correct verdict that hid a skipped verification step. These come from other papers and settings and none is the provider-filter case. What they share is that the reported number alone does not name the layer or cause behind it, a grouping the vault makes and none of those papers states. It reads as one case of the habit diagnosed in Why do safety failures remain invisible to our evaluation methods?, here applied to an attack-success column.

The figure can also run the other way. Can a poisoned validator still approve unsafe actions? reports an undefended pipeline where memory poisoning succeeds every time, against this paper's undefended zero on tool- and memory-poisoning. That excerpt names no backend, so the vault cannot say whether a provider filter, the payload form or the success criterion accounts for the difference. An open tension in ops/tensions/ holds it. Separately, Can a model-level filter truly contain an agent with environment access? argues that a filter is the wrong kind of control for containment even when it is visible. That pairing is the vault's reading; the review behind it does not discuss server-side filters.

What the excerpt does not say: how much safety survives on the two backends without a filter, what produced the other six blocks, or which backends those are, since only Azure GPT-5 is named. The claim here is that attribution matters, not that alignment-only backends are unsafe.

Post angle: ask of any "zero attack success" figure which layer did the blocking. This paper is at least one case where the answer is a vendor's filter.