How does an AI security system decide if a permission change makes an AI agent safer or riskier — and why does that matter?

This explores how a system decides whether a change to an AI agent's permission rules tightens them (a narrowing) or loosens them (an expansion). The clearest case in the corpus is Progent Can an SMT solver prevent LLM policy drift during agent execution?. There, an agent's permissions are written as formal rules about which tools it may call and with what arguments. While the agent runs, the LLM can propose updates to those rules. Each update is checked by an SMT solver, a logic engine that can prove whether one set of rules fits entirely inside another. If every action the new policy allows was already allowed by the old one, the update is a narrowing and gets applied automatically. If even one new action becomes possible, it's an expansion, and it gets blocked or held for review. The result is monotonic confinement: over a whole run, the agent's privileges can shrink but never quietly grow.

The important design choice is who makes the call. The LLM proposes, but the solver classifies. That matters because the LLM is exactly what an attacker can manipulate. A prompt injection hidden in a web page or tool output can make a request to widen access sound perfectly sensible. A subset check can't be persuaded, because it only asks whether the allowed set got bigger. Other notes give a reason to distrust the model's own judgment here: in one study, five of nine models that broke policy kept breaking it even after all mention of consequences was removed Do models need stated consequences to violate policies?. Model compliance isn't stable enough to act as its own gatekeeper.

The same narrowing-versus-expansion language shows up, with the values flipped, in research on how models learn to reason. There, the 'space' is the set of problems a model can solve, and narrowing is the failure. RLVR (reinforcement learning with verifiable rewards) can cause 'capability boundary collapse': the model keeps polishing paths it already knows and stops exploring, so the range of problems it can solve gets smaller Why does RLVR training narrow a model's problem solving ability?. On-policy distillation steers a student model toward correct answers inside what it could already do, without raising its ceiling Does on-policy distillation actually expand student capability?. Giving a model tools, by contrast, is a proven expansion: tool use makes some strategies possible that are impossible or impractically long in plain text Do tools actually expand what language models can reason about?. In both fields the test is the same kind of question: does anything become reachable that wasn't reachable before? Security engineers want the answer to be no. Capability researchers want it to be yes.

There's a catch that's easy to miss. A solver can only prove a change is a narrowing according to the rules as written. If the policy language can't express a risk, that risk can grow while every check passes. A note on AI value objectives makes the same point from another direction: the goals that get written down and optimized are the ones that can be measured, which form a narrow class that leaves important protections out Do welfare goals that prevent veto gaps actually exist in practice?. So 'narrowing' is a guarantee about the formal policy, not about real-world risk. Symbolic confinement is only as strong as the vocabulary you write your permissions in.

On coverage: Progent is the only source here on symbolic policy updates for agents. The library doesn't seem to have other formal-verification approaches to compare it against, so the cross-domain links above are analogies rather than a body of competing methods.