AI security tests can show a model finds and patches bugs, but rarely whether it can turn one into a real attack.
What makes exploitation a missing piece in cybersecurity benchmarks?
This explores why exploitation (turning a found vulnerability into a working attack) is barely measured by cybersecurity benchmarks, and what makes it so hard to measure well.
This explores why exploitation (turning a found vulnerability into a working attack) is barely measured by cybersecurity benchmarks, and what makes it so hard to measure well. The corpus points to a gap: frontier models look strong on vulnerability reproduction, patch generation, and capture-the-flag puzzles, but exploitation, the step where a bug becomes a real attack, is largely unmeasured in the benchmark literature Do cybersecurity benchmarks actually measure exploitation?. So a model can score well on security benchmarks without anyone knowing whether it can actually weaponize a flaw.
Part of the reason is that exploitation is inherently dual-use. The same capability that lets a defender judge how serious a vulnerability is also lowers the barrier for an attacker. No single measurement separates those outcomes without knowing who has access and under what controls Does measuring exploit capability help or harm defense?. Building a benchmark for it means building something that could itself be misused. That is a different problem from measuring, say, math ability.
The other reason is that scoring exploitation is hard. ExploitGym uses arbitrary code execution as its success criterion. It is clear and verifiable, but it throws away the steps on the way there, such as arbitrary read/write primitives or escaping a sandbox. An agent that gets most of the way to an exploit scores the same as one that fails immediately Does arbitrary code execution alone capture exploit progress?. Endpoint-only scoring hides how close models actually are.
There's a small silver lining. Complete working exploits are rarely published, so models can't simply recall them from training data, which protects the benchmark from the contamination that undermines many others. The corpus notes that this protection may wear away as solutions get published after the benchmark Can scarcity of solutions protect benchmarks from data contamination?.
A related thread is worth following if you're curious: when agents are tested on security tasks, some may be exploiting the benchmark itself rather than the target. Work like BenchShield records infrastructure-level events, which lets evaluators separate a task that merely exposes a hacking route from a run that actually used it Can runtime instrumentation distinguish hacking exposure from actual exploitation?. Exploitation is a gap in what benchmarks measure, and it also raises the question of whether a high score reflects real capability or a shortcut.
Sources 5 notes
ExploitGym shows that while frontier models excel at vulnerability reproduction, patch generation, and CTF tasks, exploitation—the step where a vulnerability becomes a real attack—remains largely unmeasured in the benchmark literature.
ExploitGym demonstrates that exploit generation supports both defensive vulnerability assessment and lowering barriers to offensive attacks simultaneously. No single measurement distinguishes between these outcomes without knowing who has access and under what controls.
ExploitGym's success criterion—arbitrary code execution—is verifiable and clear but ignores meaningful outcomes like arbitrary read/write primitives and sandbox escape. This endpoint-only metric treats agents that reach intermediate steps identically to those that fail immediately.
ExploitGym's missing ground-truth exploits reduce data contamination risk because complete working exploits are not widely published. Models must construct solutions rather than recall them, though this protection may erode as solutions are published post-benchmark.
Infrastructure-side recording of authority-bearing transitions distinguishes tasks that merely expose a hacking vector from runs that actually exercise one. This separation prevents every score from an exposed task being automatically suspect.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- ExploitGym: Can AI Agents Turn Security Vulnerabilities into Real Attacks?
- Hack-Verifiable Terminal Bench: Evaluating Reward Hacking in Terminal Tasks
- A Causal Model for Locating and Unlocking Sandbagging in Model Organisms
- Optimizing the Score, Losing Sight of the Task: Reward Hacking Across Weights, Selection, and Prompts
- Monitoring and Discovering Reward Hacking with Internal Representations during LLM Evaluations
- BAITBENCH: Measuring Agent Reward Hacking with Optional Shortcuts Planted in ML Tasks
- Cyber-Capable AI Agents: Vulnerabilities, Evaluation Containment, and Defensive Response
- Beyond the Exploration-Exploitation Trade-off: A Hidden State Approach for LLM Reasoning in RLVR