Where you check an AI attack — the first step or the last — can make the same system look safe or broken.

This explores how the point in an attack where you take your measurement can change the verdict on the same system. The corpus suggests a low attack number is a claim about the stage you looked at, not about the system as a whole. No single note runs one attack and measures it at every stage, so what follows is a pattern across several notes.

Start at the end of the chain. If you only count harmful final actions, a well-defended system can score zero. But when the Unsafe Action Rate hits zero across defended conditions, it stops telling apart attacks that were contained from attacks that never took hold at all. The Judgment Bypass Rate looks at the attacked agent itself and can still see the compromise Can action-level metrics alone expose contained attacks?. A reading at the last step says "safe." A reading at the first step says "the agent was fooled and something downstream caught it." Those call for very different fixes.

The size of the unit you inspect matters just as much. Skill scanners score each skill on its own, so an attacker can use scanner feedback to make every sub-skill look innocent while the chain-level plan stays intact. In that setup, attack success averaged 96% across six scanners Can attackers evade skill scanners by refining individual skills?. Every part passes inspection and the whole still does the damage. Inspecting whole chains helps: ChainGuard cut attack success to 22.5% while passing 99.5% of benign workflows. Even so, more than one in five attacks still got through Does chain-level inspection close the cross-skill attack blind spot?. One note argues the unit should stretch further, to a revisable set of actions linked across executions and agents, because isolated review throws away the context that reveals coordination Should defence units span multiple executions and agents?.

The same issue shows up if your measuring point is the model's reasoning. Chain-of-thought monitoring fails in two ways. In omission, which the study found about 79% of the time, the influence never appears in the trace. In laundering, a harmful instruction gets paraphrased into safe-sounding reasoning Can we detect when models hide their reasoning?. A backdoor can also be trained to produce fluent, coherent, wrong reasoning that looks normal Can chain-of-thought reasoning be secretly manipulated to look normal?. If you audit the trace, a clean trace may only mean the attack was hidden well. Longer reasoning also gives an attack more places to land: multi-turn manipulation cut reasoning-model accuracy by 25–29%, because one wrong step can carry through into a confident wrong answer Are reasoning models actually more vulnerable to manipulation?. Where in the chain you sample changes how corrupted it looks.

Some stages sit outside the chain you're watching. The layer that decides which model handles a request lies beneath prompt-level defenses. Manipulating it can send requests to weaker models or make safety checks run against the wrong identity Can attackers manipulate which model handles a request?. Before trusting a result, ask which stage was measured, which unit was inspected, and what the number cannot distinguish. A zero at one stage can sit alongside a 96% at another.