Do AI-agent attacks slip past security scanners by being tuned to each one, or by exploiting a flaw scanners share?

In the corpus's clearest case, both matter, and they stack. ColluSkill, an attack that hides malicious intent across several AI agent skills, reaches 96% average attack success against six different scanners. It gets there by using each scanner's own feedback to refine every sub-skill until it looks harmless Can attackers evade skill scanners by refining individual skills?. The tuning is what does the evading. The recipe is portable, though: it is a feedback loop, not a trick built for one scanner's quirks.

Underneath the tuning sits a structural blind spot that needs no tuning at all. Scanners that inspect skills one at a time can't see an attack split into interdependent pieces. Each skill looks harmless alone, and only their ordered execution carries out the attack Can skill scanners miss attacks hidden across multiple skills?. The scanner's unit of inspection is smaller than the attack's unit of action. The corpus doesn't say how much of the 96% comes from that blind spot and how much from feedback refinement, so the split is unknown.

The same question applies to defense numbers. ChainGuard's reported 22.5% attack success comes with no evidence that the attacker refined its strategy against ChainGuard's feedback, so it may reflect a weaker attacker than a determined one Does ChainGuard's defense hold against adaptive attacks?. That fits a wider point: formal limits on a system's vulnerability don't predict what attackers find. Actual exposure depends on where the scoring errors sit and how well the attacker searches Can distance alone rank which substrates resist reward hacking?. My read is that this is the reward-hacking pattern applied to security. Reward hacking arises when something optimizes against a signal that only partly captures the real goal Does reward hacking always stem from the same failure?. A scanner is such a signal, so an attacker who can query it is optimizing against it.

Some attacks work with no tuning to a specific defense, and how well they hold up varies by type. Pretraining poisoning at 0.1% of data survives standard safety alignment for denial-of-service, context extraction, and belief manipulation, but jailbreak-style poisoning gets suppressed How much poisoned training data survives safety alignment?. Manipulative multi-turn prompts cut reasoning models' accuracy by 25-29%, because longer reasoning chains give a single wrong step more chances to spread Are reasoning models actually more vulnerable to manipulation?. Routing attacks sit below the prompt-level defenses entirely, sending requests to weaker models or making safety checks run on the wrong identity Can attackers manipulate which model handles a request?.

The practical upshot is to ask whether the attacker got to see the defense's feedback. A low success rate against an attacker who never saw the defense says little. A high one from an attacker who could iterate against it says little about general robustness in either direction. The corpus has no head-to-head test of tuned versus untuned versions of the same attack, so that comparison is a gap.