Progent: Securing AI Agents with Privilege Control

AI agents interact with external environments through tool calls, exposing them to attacks like indirect prompt injection that can trigger unauthorized actions. Securing these agents is challenging: they behave autonomously and probabilistically, security requirements evolve depending on the user’s task and execution state, and there is an inherent tradeofff between security and utility. In this work, we introduce Progent, a novel framework that secures AI agents via privilege control. Progent represents privilege as a security policy consisting of symbolic rules over tool names and arguments. These rules specify which tool calls are allowed for task completion and which unnecessary ones are blocked for security. Every tool call is checked against such a policy through a deterministic procedure, enforcing the principle of least privilege. To handle diverse user tasks and evolving execution contexts, an LLM automatically generates the initial policy from the user’s task and updates it during execution as new information arrives. Each proposed update is determined by an SMT solver to be either a narrowing (applied automatically) or an expansion (requiring explicit approval), ensuring that the agent’s effective action space can only shrink without approval (monotonic confinement). This deterministic update mechanism preserves utility and prevents silent privilege escalation, even when adversarial inputs are present. Our evaluation on popular benchmarks (i.e., AgentDojo and ASB) shows that Progent significantly reduces attack success rates while maintaining high utility. We further validate Progent’s practicality by showcasing its effectiveness in real-world agent frameworks such as LangChain and OpenAI Agents SDK.

Introduction. AI agents have emerged as a promising platform for general and autonomous task solving [41, 46, 47, 56]. At the core of these agents is a large language model (LLM), which interacts with the external environment through diverse sets of tools [39, 40]. For instance, a personal assistant agent uses email toolkits [20] to manage emails, and a coding agent uses code interpreters and the command line [47].

Security Risks in AI Agents. The ability to interact with external environments makes AI agents powerful, enabling them to perform diverse tasks across domains such as communication, scheduling, and finance. However, this same ability also creates a large attack surface. Recent research has raised serious security concerns about AI agents [15, 28, 52], where attackers exploit the agent’s interaction with the environment by injecting adversarial instructions into external data sources. When the agent retrieves such data via tool calls, the injected instructions can redirect the agent to perform dangerous actions, such as unauthorized financial transactions [11] or data exfiltration [29]. This class of attacks is known as indirect prompt injection [14, 30].

Challenges for Securing AI Agents. Securing AI agents is challenging for several reasons. First, agent behavior is inherently nondeterministic: an LLM processes unstructured natural language and may make different decisions across runs. In contrast, security requires deterministic enforcement that holds regardless of how the LLM reasons, even under adversarial influence. Reconciling these two worlds is non-trivial. Second, agent security depends on both the user’s task and the information the agent gathers during execution. Different tasks require different tools and arguments, and the relevant context evolves at runtime as the agent interacts with the environment. A defense must track these changes automatically, since involving the user in every security decision is impractical and negates the benefit of an autonomous agent. Finally, agents must sometimes expand their privileges to complete benign tasks. Yet, adversaries also seek to exploit the same mechanism to induce harmful actions, and both legitimate and adversarial expansion requests arrive through the same channel. Distinguishing legitimate from adversarial expansions, without sacrificing security or utility, is a fundamental challenge. We elaborate on these challenges with a running example in Section 2.

Our Work: Progent. We introduce Progent, a novel framework that secures AI agents via privilege control. Our key idea is to enforce the principle of least privilege: allow only tool calls essential for task completion and block unnecessary ones. We focus on tool calls because all external effects of an agent occur through them, and they expose a structured interface (tool names and typed arguments), unlike the agent’s unstructured reasoning. This makes tool calls a natural point for deterministic security enforcement. Progent leverages a privilege control policy consisting of symbolic rules over tool names and arguments. These rules specify which tool calls are allowed and which should be blocked. Every tool call is checked against such a policy, which deterministically decides whether it is permitted. To handle context-dependent security requirements, Progent utilizes an LLM to automatically generate the initial policy from the user’s task and updates it as new information arrives during execution. This allows security enforcement to adapt to both task and execution context while minimizing manual effort. However, introducing an LLM for policy construction introduces non-determinism, creating a fundamental tension between automation and deterministic enforcement. To address this, we further propose that each policy update be checked by a deterministic SMT solver as either a narrowing (applied automatically) or an expansion (requiring explicit approval). This ensures that the agent’s effective action space can only narrow without approval, a property we call monotonic confinement. As a result, even if the LLM is manipulated by adversarial inputs, this deterministic check prevents any silent privilege escalation. At the same time, it preserves utility by allowing controlled privilege expansion when explicitly authorized.

Main Contributions. In summary, our main contributions are: • A privilege control approach that leverages symbolic policies over tool names and arguments, with deterministic enforcement at the tool-call level and an SMT-based procedure to determine whether policy updates are expansions or narrowings (Section 4). • An LLM-based policy generation and update mechanism that automatically derives context-aware policies from the user’s task and execution state, with formal monotonic confinement guarantees that prevent silent privilege escalation (Sections 5 and 6).

Related work. Security Policy Languages. Enforcing security principles is challenging and programming has been demonstrated as a viable solution by prior works. Binder [12] uses Datalog-style inference for authorization and delegation in distributed systems, while Sapper [27] introduces hardware-level checks for timing-sensitive noninterference. At the cloud and application level, Cedar [8] provides a domain-specific language for expressing fine-grained authorization policies, and major cloud platforms such as AWS [1], Microsoft Azure [33], and Google Cloud [13] also offer established authorization policy languages. These approaches show how programmatic policy enforcement has matured across diverse security domains, yet existing policy languages are not suited to the dynamic behavior of AI agents. This gap makes agent-oriented policies such as Progent a natural next step.

System-Level Defenses for Agents. Developing system-level defenses for agentic task solving is an emerging research field. IsolateGPT [54] introduces an agent architecture that isolates execution across applications and requires user confirmation for potentially dangerous operations (e.g., cross-app communication or irreversible actions), while f-secure [51] enforces information-flow controls through manually assigned trust labels that propagate during agent execution. AirgapAgent [2] focuses on access control over private data, ensuring agents only use task-relevant information, but does not control the agent’s malicious behavior beyond data access. CaMeL [10] extracts control and data flows from trusted user queries to prevent untrusted data from affecting program flow, but requires modifying the agent’s internal architecture and affects utility. FIDES [7] uses dynamic taint tracking with confidentiality and integrity labels to regulate data flows through agent operations, but requires up-front label assignment to data sources. Conseca [44] and DRIFT [25] also propose generating policies for agents, but both rely on an LLM for generating the policies without deterministic verification. Progent’s policies are structured to support an SMTbased expansion check that deterministically classifies each update as an expansion or a narrowing of privileges. In Progent, even if the LLM-generated policy update is manipulated by adversarial inputs, the expansion check prevents any silent privilege escalation, enabling monotonic confinement guarantees that neither Conseca nor DRIFT can provide. The privilege-control approach introduced by Progent complements these defenses. Its modular design enables integration into existing agent implementations with minimal changes, potentially lowering adoption barriers, whereas many other system-level defenses require substantial architectural modifications.

Method. 2.2 Progent: Defense via Privilege Control We now describe how Progent addresses the above challenges.

Policy-Based Security Enforcement. Progent operates at the tool-call level, because all agent actions with external effects flow through them. Moreover, the tool-call interface, consisting of a tool name and typed arguments, is structured, unlike the unstructured text the agent reasons over. This makes tool calls a natural enforcement point, enabling Progent to enforce security deterministically despite the agent’s non-deterministic behavior (Challenge I). Progent introduces a privilege control policy that describes the set of tool calls the agent is allowed to issue or which should be blocked. A policy is a list of rules; each rule targets a specific tool and specifies conditions on its arguments to determine whether calls to that tool are allowed or not allowed. In our example (Figure 1b), the policy P4 contains four “allow” rules, meaning that a tool call is allowed if its name matches the rule and its arguments satisfy the specified condition. For instance, the rule for read_emails allows calls only when recent=20, while the rule for send_slack_msg allows calls only when to=alice. Rules can specify concrete values (such as 20), wildcards (*), and regular-expression patterns, making the policy expressive enough to capture fine-grained constraints.

Apart from “allow” rules, Progent also supports “forbid” rules that explicitly block tool calls matching certain patterns, as well as various other features. We provide a full definition of Progent’s policies in Section 4.1. When the agent issues a tool call, Progent matches it against the current policy’s rules and deterministically allows or blocks it; calls that match no rule are blocked by default. At step 4, P4 contains no allow rule for send_email, so Progent deterministically blocks the malicious call send_email(to=eve@evil.com,body=(inbox) ) and returns a fallback error to the agent explaining the denial, so the agent can adjust and continue with the user’s task. At step 5, the check is against P5 = P4; the benign call send_slack_msg( to=alice,body=(summary)) matches the Slack rule in P5, so it is allowed and the task completes. Because this check is purely symbolic over structured policy rules, every security decision is deterministic, directly addressing Challenge I.

LLM-Based Policy Construction and Update. Because agent security depends on context (Challenge II), Progent maintains a per-step policy that evolves as new information arrives: at step i, the gating privilege control policy is Pi, which enumerates exactly which tool calls the agent may issue at that step. The initial policy P1 is derived from the user’s task: for our running example, P1 admits only search_emails with the specific sender and subject. An LLM-based updater reasons about both the task context and the execution context, automating policy management that would otherwise require manually writing per-step rules for every possible user request. For i> 1, the updater proposes a new policy Pi based on the tool return at step i−1 and the previous policy Pi−1. After step 1, the returned TODO reveals that the task requires additional tools, so the updater proposes P2, which adds allow entries for read_emails, get_slack_info, and send_slack_ms g. After step 2, the handle alice is now known, so the updater proposes P3, which narrows the Slack entry to send_slack_msg( to=alice,body=*). After step 3, the returned emails contain an injection attempting to introduce send_email, but the policy does not change: P4 = P3. How this is ensured is described next.

Deterministic Control of Policy Expansion and Narrowing. Policy updates can either expand or narrow the set of allowed tool calls. Narrowing is safe, since it only removes permissions. Expansion, however, is sensitive: it may be necessary for utility (as in P1 →P2, where the task requires more tools) but can also be caused by an attack (as in step 3’s injection). To handle narrowing and expansion differently, Progent judges every proposed update before applying it.

Conclusion. We propose Progent, a framework that secures AI agents via privilege control at the tool-call level. Progent introduces a privilege control policy consisting of symbolic rules over tool names and arguments, enforced deterministically with no LLM in the decision loop. To handle diverse tasks and evolving execution contexts, an LLM automatically generates and updates the policy, while a deterministic SMT-based check determines whether each update is a narrowing (applied automatically) or an expansion (requiring approval), guaranteeing monotonic confinement. Our evaluation on AgentDojo and ASB demonstrates that Progent significantly reduces attack success rates while preserving high utility, and integration with real-world agent frameworks such as LangChain and OpenAI Agents SDK confirms its practicality as a non-intrusive defense-in-depth layer.

Limitations. User Mistakes. Although Progent incorporates deterministic SMTbased checking, user mistakes may still jeopardize its security guarantees. If users incorrectly approve widening updates or give ambiguous initial tasks, Progent may permit policies that exceed minimal privilege. Our mechanisms of policy updates, enforcing invariant constraints, and requiring explicit approval for potentially unsafe changes mitigate but cannot eliminate these risks. In Appendix E, we empirically evaluate Progent against user-introduced mistakes using an adaptive attack, showing that Progent substantially reduces ASR and retains meaningful protection despite possible human misjudgment.

Defense Scope. Progent focuses on protecting against malicious tool calls that fall outside the invariant boundaries. As such, Progent does not tackle attacks that influence only the agent’s textual outputs without triggering dangerous tool calls. Such text-to-text attacks are more aligned with model-level risks that have been extensively studied by existing model security works. For agents, Progent serves as an effective defense-in-depth layer that complements other defenses while requiring minimal code changes.