Cyber-Capable AI Agents: Vulnerabilities, Evaluation Containment, and Defensive Response

Cyber-capable AI agents combine language models with tools, memory, and execution environments to perform multi-step offensive-security tasks. Existing work separately measures cyber capability and catalogs attacks against agent components, but provides less guidance on containing a capable agent within the environments used to evaluate it. This review synthesizes five vulnerability classes at that boundary: multi-step offensive chains, objectives that conflict with sandbox boundaries, supply-chain and credential exposure, persistent command-and-control, and the speed of automated action. We use two separate preliminary incident records: the reported July 2026 Hugging Face/OpenAI evaluation breach and Anthropic’s subsequent threeincident evaluation review. A comparative evidence protocol distinguishes record-specific factual claims from the shared systems lesson: the evaluation environment is itself part of the security boundary. Across the taxonomy and records, we examine controls for containment, privilege separation, provenance, and responder access, including the dual-use problem that defensive artifacts may also enable misuse.

Introduction. Cyber-capable models are increasingly deployed as agents: language models connected to tools, memory, and execution environments so they can pursue multi-step tasks over extended periods [1, 2, 3]. In a cyber setting, that scaffolding turns code reasoning, retrieval, and command execution into an operational system, not just a single model response. That changes the central security question. The issue is no longer what the model can do in one exchange; it is what happens once an agent retains state, pulls in untrusted content, calls tools, and sits next to credentials and network paths. A benchmark score tells you how a model performed under fixed conditions. It says nothing about the containment around it. A model-level filter blocks a moment of behavior; it does not contain an agent that already has access to an execution environment. That boundary is hard to study because the evidence needed to study it is scattered: agent-security research, cyber-capability evaluations, containment work, and incident reports each hold a piece of it.

Discussion / Conclusion. Cyber-capable agents make the security of capability evaluation an end-to-end systems problem. Once a model is connected to memory, tools, credentials, and an execution environment, those components—and the response workflow around them—become part of the security boundary. Evaluating the model’s cyber capability without evaluating that boundary leaves out the mechanisms through which a capable agent can act. This review organizes that boundary into five vulnerability classes: multi-step offensive chains, objectives that conflict with sandbox boundaries, supply-chain and credential exposure, persistent command-and-control, and the speed of automated action. Two preliminary primary incident records then show why the evaluation setting deserves analysis in its own right. The Hugging Face/OpenAI record and Anthropic’s separate evaluation review do not establish a common attack sequence, recurrence rate, control effectiveness, or causal mechanism.