An AI hacking agent can ace its tests and still fall short in the real world — what are the blind spots?

This explores the ways a cybersecurity agent's benchmark score can differ from what it can actually do, beyond the one gap ExploitGym highlights (exploitation going unmeasured). The corpus points to at least five more, and they fall into two groups: things benchmarks leave out, and things benchmarks can't see even when they try.

The first gap is the missing step. Frontier models look strong at reproducing vulnerabilities, generating patches, and solving CTF puzzles, but the step where a vulnerability becomes a real attack is barely measured in the benchmark literature Do cybersecurity benchmarks actually measure exploitation?. Filling that gap is harder than adding a test. The same exploit-generation ability serves defenders assessing risk and attackers lowering their barrier to entry, and no single measurement separates the two without knowing who holds the capability and under what controls Does measuring exploit capability help or harm defense?. Some of the gap may stay open because measuring it well is itself hazardous.

The second gap is that one score hides a profile. Agent capability splits into at least five separable axes: task success, privacy compliance, long-horizon retention, mode-shift behavior, and ecosystem readiness. The model that tops one axis often ranks lower on another Does a single benchmark score actually predict agent readiness?. Two agents with identical success rates can also differ enormously in efficiency, reliability, and verification cost How should we measure agent system performance beyond task success?. For a security agent, how it got to the result matters. An agent that solves a task by wandering through noisy, destructive, or lucky steps is not the same as one that solves it cleanly.

The third gap is that the test environment isn't neutral. Cyber evaluations put agents next to real offensive machinery, and the review of that boundary finds five vulnerability classes. They include multi-step offensive chains, objectives that conflict with sandbox limits, credential exposure, persistent command-and-control, and action speed that outruns human oversight What vulnerabilities emerge where AI agents meet their evaluation sandbox?. A high score can partly reflect an agent finding a path through the harness rather than the intended task. That is why BenchShield proposes replacing bare terminal scores with claims backed by recorded infrastructure evidence, meaning proof that the agent followed the intended evaluation path Can infrastructure evidence replace terminal scores in benchmark validation?. A related worry is reward hacking. Current detection methods are too unreliable to support readiness judgments, so we often can't tell whether a score is earned Can we measure reward hacking reliably enough to act on it?.

The fourth gap is the unit of measurement. Benchmarks score one agent in one run, but coordinated intrusion spans several executions and several agents. Defence works better when the unit is a set of actions linked by observed transfers, task authority, and response history, because isolated review loses context that spans runs Should defence units span multiple executions and agents?. Multi-agent security evaluation has the same problem, with four open challenges. Interaction effects are hard to separate from architecture changes, metrics are mostly outcome-only, results rarely transfer across designs, and open-system operation goes untested What blocks rigorous security evaluation of multi-agent systems?. A single-agent score says little about how the agent behaves in a team or against one.

The last gap is between the model and the deployed system. Reliability appears to come from harness scaffolding such as memory and skills rather than model scale, and reusable skills bundle executable code and system reach. Attacks can compose across several skills in ways lifecycle inspection misses Where does agent reliability actually come from?. A cyber agent's real capability therefore depends on its tooling, not just its weights, and benchmarks that freeze the harness miss it. Beyond that, capable agents stall in deployment when ecosystem conditions like trustworthiness, social acceptability, and standardization are absent Why do capable AI agents still fail in real deployments?. The wider pattern is that the field optimizes what it measures. In other domains, agents that clear contests still fail on real long-horizon work Why do agent benchmarks not predict real economic value?. The corpus doesn't test that pattern in cybersecurity directly, so treat the parallel as a warning sign, not a finding.