If an AI system reports zero successful attacks but someone else's filter did the blocking, is it actually safe?

This explores whether a pipeline with no defenses of its own can call itself safe when it reports zero successful attacks, and the filter that blocked them belongs to someone else. The corpus says no. It can claim that attacks didn't get through in this setup, which is a weaker claim than saying the system is safe. In one multi-agent pipeline tested across different backends, 54 of the 60 blocks came from Azure's cloud filter, not from the application Where do safety wins come from in multi-agent systems?. The pipeline was inheriting its safety, and that inheritance stays invisible until someone changes the backend.

Part of the problem is how results get reported. If you only publish the outcome (attacks succeeded: 0%), nothing shows which layer did the blocking. The vault notes that most published attack and defense numbers don't say whether they were measured behind a provider-side filter Which attack and defense numbers came from filtered backends?. A reader can't tell whether a low attack rate reflects the model, the application, or the cloud provider's screening. Comparing numbers across papers is shakier than it looks, because some results may be measuring the filter rather than the system.

Even when the filter is real, it covers less than it seems to. A filter judges one output at one moment, but an agent's risk spreads across memory, retrieved content, tool calls and environmental reach, so blocking a moment of behavior doesn't contain the agent Can a model-level filter truly contain an agent with environment access?. Inside a planner-worker-verifier-synthesizer pipeline, five internal hops (tool to worker, memory to worker, and so on) get no inspection, and existing defenses mostly watch only user input Do internal agent hops in pipelines need security monitoring?. An injection hidden in a tool result or in memory can travel downstream without passing the checkpoint the filter guards. Per-action checks also can't express rules about sequences, such as two individually fine actions that together violate a constraint Can stateless checks ever catch sequence-level constraint violations?.

The filter's own position is also something an attacker can target. The layer that decides which model handles a request is an attack surface, and manipulating it can send requests to weaker models or make safety measures act on the wrong identity Can attackers manipulate which model handles a request?. A zero attack rate measured against one routing path says little about the path an adversary would choose.

To earn a safety claim, the application has to own some protection and show where each block came from. That could mean reporting results with the provider filter on and off. Owned defenses don't have to be heavy. Retrieval-layer methods can catch RAG poisoning without retraining Can we defend RAG systems from corpus poisoning without retraining?. Stating a rule isn't enough either: in one test, explicit authorization boundaries protected tests from modification only when paired with restricted tools Can explicit authorization boundaries prevent agents from modifying protected tests?. Your own gates should be checked too. One sanitizer gate leaked 23.4% of what it was meant to stop, which is why re-scoring what it keeps is proposed Should sanitizers re-score their compressed output before passing it?. The takeaway is that a filter can block attacks for you, but you can't claim the safety unless you know which layer is blocking them and what happens when that layer is gone.