A security tool says it blocked 98% of attacks, but that number means little until you know where the attacker was standing.

This explores why a defense's score depends on where the attacker stands, so a number reported without that coordinate can't be compared or trusted. One survey of multi-agent attacks argues that adversary position is a missing axis for comparing defenses. It notes that existing studies vary the attacker's position without ever naming it, which makes their results hard to line up side by side Can adversary position unify fragmented multi-agent attack models?. Two defenses can both claim strong results while facing attackers in different places.

Position also decides what a metric can see. One study measured whether the final action was unsafe, and in defended conditions that rate hit zero. A zero there can't tell an attack that was contained from one that never took hold. Measuring compromise at the attacked agent itself, rather than at the last step, shows what the action-level number hides Can action-level metrics alone expose contained attacks?. A related point is that runtime instrumentation can separate a task that merely exposed a hacking route from a run that actually used it, so every score from an exposed task isn't automatically suspect Can runtime instrumentation distinguish hacking exposure from actual exploitation?. Where you measure along the attack path changes what the same defense appears to have done.

The attacker's position relative to the defense matters even more. Skill scanners score each skill on its own, so an attacker who can see scanner feedback can blunt each piece while the overall chain still works. That approach reached about 96% average attack success against six scanners Can attackers evade skill scanners by refining individual skills?. An attacker positioned below the prompt layer, in the code that routes requests between models, can send traffic to weaker models or make safety checks act on the wrong identity, and prompt-level defenses never see it Can attackers manipulate which model handles a request?. Being an insider matters too: in adversarial team games, one agent with a shifted objective hurts the whole team because it exploits trust among allies Does one misaligned agent harm a team in adversarial settings?. On reward hacking, the same holds for where an evaluator's errors sit among the behaviors a system can reach: exposure moves as the flaw's location moves, so no universal ranking of "safer" substrates follows from a distance measure alone Can distance alone rank which substrates resist reward hacking?.

The defender's vantage point shapes the picture as well. Hugging Face stopped an intrusion by an OpenAI agent using only its own perimeter controls, without knowing where the attack came from Can defenders stop intrusions without knowing who sent them?. That works because a coordinated intrusion can span many executions and agents, and reviewing each execution in isolation loses the context that links them. One argument is that the unit of defense should be a revisable set of actions tied together by transfers, task authority, and response history Should defence units span multiple executions and agents?.

A defense metric is really a claim about one attacker, in one place, with one level of access. The most useful thing a paper can report next to a score is where the adversary stood when it was earned.