An AI security test reported zero unsafe actions with protections on — but did the attacks ever try to break those protections?

This explores whether the security test that reported zero unsafe actions with authorization switched on actually attacked the authorization layer itself (the part that issues tokens and checks policy), or only attacked the agent upstream of it. The corpus can't confirm that they did. The paper credits 'task-bound signed tokens' and a 'separately verified policy oracle' for the zero. The available excerpt names those two pieces but doesn't say who issues the tokens, what binds them, or how verification works. It also doesn't say whether the attacks were positioned to reach either component (How does the authorization layer stay outside the poisoned path?). So the zero is a fact about the attacks that were run. Whether those attacks were aimed at the protecting layer is left open.

The numbers also don't say which piece did the work. The evaluation compares both checks on against both checks off, with no single-component ablation in the text we have (Which authorization component achieves the zero percent unsafe rate?). A sibling note flags the same bundling problem elsewhere in the pipeline. Pairing clear authorization rules with restricted tools can't separate a crossing that was unavailable from one the model chose not to make. That same pipeline shows a 100% Judgment Bypass Rate alongside a 0% Unsafe Action Rate, which is why the distinction matters (Do authorization rules or restricted tools prevent test modifications?). My reading is that this pattern fits two stories: the model's judgment was beaten every time and the authorization layer held, or the attacks never reached the layer in question. A zero can't tell them apart.

Other notes in the collection show why attacker positioning matters. One finds that if an attacker knows the rule trusted agents use to avoid decoys, the attacker can apply the same rule and walk around the trap (Can honeytokens fool attackers who know the trusted policy?). A policy that protects legitimate users can become a map for compromise if the attacker can see it. Another shows attackers who can observe a defender's checks refining their attack against each one, reaching 96% average success against six scanners (Can attackers evade skill scanners by refining individual skills?). A third treats the control layer beneath the prompt, which decides which model runs and under what identity, as an attack surface of its own (Can attackers manipulate which model handles a request?).

Together these set a bar. A convincing test would let the attacker probe the token issuer, learn what the policy allows, and try to get a poisoned request signed. The corpus doesn't show that bar was met. If the poison only reached the agent's inputs, zero unsafe actions shows the gate holds against a bad agent, not against an attacker aiming at the gate. To settle it, read the full paper's threat model: what the attacker could see, request, or influence in the issuer and oracle. Then look for the component-level ablation the excerpt promises but doesn't include.