Can an SMT solver prevent LLM policy drift during agent execution?
This explores whether a deterministic symbolic solver can catch when an LLM's security policy inadvertently expands the agent's capabilities, and whether that check actually blocks privilege escalation under adversarial conditions.
Progent secures AI agents by enforcing a "privilege control policy," defined as "a security policy consisting of symbolic rules over tool names and arguments," checked against "every tool call... through a deterministic procedure, enforcing the principle of least privilege." Rules are "allow" or "forbid" entries that can specify "concrete values... wildcards... and regular-expression patterns," and a tool call that "matches no rule" is "blocked by default." In the paper's running example, the policy denies a malicious send_email call that adversarial content injected into the agent's context tries to trigger, and allows a benign send_slack_msg call under the same policy, because the check is "purely symbolic over structured policy rules" and does not depend on the agent's own, manipulable reasoning.
Because "different tasks require different tools and arguments" and "the relevant context evolves at runtime," Progent has an LLM generate the initial policy from the user's task and propose updates as execution proceeds — for instance, after a tool result reveals a TODO requiring more tools, or reveals a specific recipient that narrows a rule to that recipient. Letting an LLM write and revise the policy "introduces non-determinism," so Progent adds a second, deterministic check on top: "each policy update [is] checked by a deterministic SMT solver as either a narrowing (applied automatically) or an expansion (requiring explicit approval)." The paper names the resulting guarantee "monotonic confinement": "the agent's effective action space can only narrow without approval," so that "even if the LLM is manipulated by adversarial inputs" — as when injected email content tries to introduce a new send_email capability — "this deterministic check prevents any silent privilege escalation," while genuine, approved expansions still go through.
The design lands on a split already in the library's Can memory poisoning compromise decision-making even with authorization layers?: a reasoning component can be fooled every time while a separate, non-reasoning check still blocks the resulting action. That paper measures a token-plus-oracle layer at the point of execution without saying how the oracle itself decides; Progent proposes one way to build such an oracle — a symbolic allow/forbid policy plus an SMT-based narrowing/expansion check — and locates its guarantee in the update mechanism, not only in the final per-call check. The two differ in grain: the authorization layer passes or blocks a single step, while Progent's policy is stateful across the run (the paper's P1 through Pi notation), evolving as context accumulates, which moves toward the trajectory-level view in Can step-by-step approval miss harmful behavior patterns? — though Progent's own enforcement is still a call-by-call match against the current policy, not a check over the sequence as a whole.
The excerpt reports the security/utility result only in general terms — Progent "significantly reduces attack success rates while maintaining high utility" on the AgentDojo and ASB benchmarks — without the exact percentages, so the size of that tradeoff is not established here. The paper's own Limitations section is explicit that the guarantee covers silent escalation, not human error in the approval step: "If users incorrectly approve widening updates or give ambiguous initial tasks, Progent may permit policies that exceed minimal privilege." It also scopes the defense to actions: Progent "does not tackle attacks that influence only the agent's textual outputs without triggering dangerous tool calls." Monotonic confinement, then, is a guarantee that privilege cannot expand without a recorded approval, not a guarantee that approval is always given correctly, and not coverage for harm that never passes through a tool call.
Inquiring lines that read this note 18
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
How should authorization govern AI agent autonomy?- What entitlements do users retain that operators cannot override?
- What mechanisms enforce authority link constraints when agents run?
- How much does capability enforcement cost legitimate agent tasks?
- What makes certain agent actions categorically high-risk and unrecoverable?
- Does formal scope checking actually prevent compromised agents from exceeding boundaries?
- What counts as a narrowing versus an expansion in symbolic policy space?
- How do models prevent lower-level messages from spoofing higher-authority instruction sources?
- How does symbolic matching on tool names prevent prompt injection attacks?
- Can defenses that secure only control flow miss attacks on data values?
- Why does isolating the planner fail to stop value manipulation attacks?
Related concepts in this collection 3
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Can memory poisoning compromise decision-making even with authorization layers?
When authorization systems add signed tokens and policy oracles to an AI pipeline, does this stop attackers from poisoning the agent's judgment about whether to approve actions, and what actually prevents unsafe execution?
the same split, a fooled reasoning component contained by a separate deterministic check, for which Progent proposes one construction
-
Can step-by-step approval miss harmful behavior patterns?
If each action an agent takes passes its individual safety check, can the overall sequence still violate system constraints? This matters because per-action inspection may miss harms that emerge only across time or composition.
Progent's policy evolves across the run but its own check still acts call-by-call, short of this thesis's trajectory-level claim
-
Should safety harnesses be customized for each deployment?
Can a single safety harness design work across different models and domains, or does each deployment need its own tuned version? Understanding this matters for scaling AI safety practices efficiently.
qualifies: argues one-size-fits-all harnesses over-block or miss domain safety relations, questioning whether Progent's generic allow/forbid policy generalizes across deployments
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Progent: Securing AI Agents with Privilege Control
- Defeating Prompt Injections by Design
- Trust propagation and structural containment in Multi-agent LLM pipelines
- Design Patterns for Securing LLM Agents against Prompt Injections
- SoK: When Safe Agents Fail Together: The Security of Multi Agent LLM Systems
- Securing Agentic AI: From Per-Action Checks to Trajectory Assurance
- ChannelGuard: Safe Models Do Not Compose into Safe Multi-Agent Systems
- BenchShield: Formal Model-Backed Instrumentation for Reward Integrity in LLM-Agent Evaluation Infrastructure
Original note title
Progent's SMT solver, not the LLM, classifies each policy update as a narrowing or an expansion to guarantee monotonic confinement