How can autonomous agents act safely within defined boundaries?
Explores how formal scope records might constrain agent autonomy while preserving practical action, balancing security against user confirmation fatigue.
For "complex or multi-step goals (e.g., booking a trip, drafting a business plan, coordinating a software rollout)" where "requiring explicit confirmation for every step would often be impractical," OpenAI's Model Spec proposes recording the bounds of autonomous action formally: "Scopes should be recorded in a semi-structured format — for instance, a ScopeOfAutonomy record with fields allowed_tools, latest_time, max_cost, tool_constraints, and free-text additional_details — to facilitate validation, enforcement, and auditing by the platform." Every such scope "must include an ending condition," and the Spec calls a time limit "a best practice" for that condition. A fixed list is carved out as always off-limits: "High-risk activities (e.g., hacking, deception, resource acquisition, spawning sub-agents, self-modification) are always prohibited unless explicitly authorized by applicable instructions."
The reasoning is drawn explicitly from software security: the approach "mirrors the principle of least privilege and capability-based security common in software security," and scope negotiation is likened to "a consulting firm submitting a scope-of-work." OpenAI frames this as a balance against two failure directions, not just one. Too broad, and the assistant can act "even if an out-of-scope action seems to be in the user's best interest" — "no exceptions apply" once a scope is agreed, absent explicit re-approval. Too narrow, and the cost is behavioral: "scope should also not be so narrow as to require multiple confirmations by the user for trivial actions, which could habituate the user to automatically confirming all requests" — a named mechanism for how over-asking produces the very rubber-stamping it was meant to prevent.
The ScopeOfAutonomy record plays a role similar to the authorization layer measured in Can memory poisoning compromise decision-making even with authorization layers?: both put a check on what an agent may do outside the agent's own judgment, so a fooled or compromised decision-maker still cannot execute past the boundary. The difference is evidentiary. That paper reports a measured 0 percent Unsafe Action Rate against a specific memory-poisoning attack on a specific four-agent pipeline; OpenAI's ScopeOfAutonomy is a proposed schema with no comparable test in this excerpt. The Spec's idea that scope can be "dynamically negotiated with the assistant for complex tasks" also touches How can users verify what an AI agent actually plans to do?, though the two address different moments: ScopeOfAutonomy bounds the action space before work begins, while that note's verification concerns tracking the agent's plans and actions as they unfold within whatever scope was granted.
What the excerpt does not say is how any of this is implemented or enforced — whether allowed_tools and max_cost are checked by a sidecar process, a tool-calling gate, or something else, and how a violation would be detected versus merely specified. It also leaves the exception clause for high-risk activity unspecified: activities are "prohibited unless explicitly authorized by applicable instructions," but who can grant that authorization, at what authority level, and under what review is not stated here. As with the surrounding chain-of-command material in the same document (see Can a fixed instruction hierarchy actually prevent prompt injection attacks?), this is OpenAI's design argument for how agentic autonomy should be bounded, not evidence of how autonomy is bounded in a deployed system.
Inquiring lines that read this note 30
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
How should agents balance abstract planning with concrete environmental grounding? How should authorization govern AI agent autonomy?- What entitlements do users retain that operators cannot override?
- What mechanisms enforce authority link constraints when agents run?
- What design principles should guide authorization boundaries in multiagent systems?
- Can deontic specifications replace runtime negotiation of task delegation?
- How should authorization exceptions for prohibited actions be granted?
- What five levels of autonomy can users set for AI agents in practice?
- How does user experience differ when agents use fixed versus adaptive autonomy?
- Can approval fatigue cause users to miss dangerous AI permission requests?
- Can scope negotiation prevent user confirmation fatigue in autonomous systems?
- Can institutional models constrain agent behavior across shifting data contexts?
- How do institutional models and protocols scaffold accountability in agentic AI systems?
- How can formal communication protocols like FIPA-ACL improve agent transparency?
- Why do deployed LLM agents lack identity verification at protocol level?
- How much does capability enforcement cost legitimate agent tasks?
- What makes certain agent actions categorically high-risk and unrecoverable?
- Does formal scope checking actually prevent compromised agents from exceeding boundaries?
- How do organizational rules prevent conflicts across multiple agent roles?
- What gaps exist between MOISE+ formal design and actual agent behavior?
- Can verification of agent actions after execution substitute for formal communication protocols?
- What does it mean for users to verify an agent's plans before execution?
Related concepts in this collection 6
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Can memory poisoning compromise decision-making even with authorization layers?
When authorization systems add signed tokens and policy oracles to an AI pipeline, does this stop attackers from poisoning the agent's judgment about whether to approve actions, and what actually prevents unsafe execution?
a measured authorization layer that plays the same role ScopeOfAutonomy is designed, but not shown, to play
-
How can users verify what an AI agent actually plans to do?
Stating goals to an AI agent isn't enough for mutual understanding. What mechanisms would let users check both the agent's proposed plan and whether it truly grasps the user's goals and constraints?
bears on the same negotiated-scope idea, but about tracking plans in progress rather than bounding the action space in advance
-
Do overseers focus on agent execution or action safety?
When humans oversee AI agents, do they check whether the agent followed its plan, or whether the action itself is safe? A study of computer-use agent oversight found this distinction matters even when intervention timing is correct.
contrast: that oversight checks plan-following, not safety, where ScopeOfAutonomy's high-risk list is a safety bound regardless of plan-following
-
Can a fixed instruction hierarchy actually prevent prompt injection attacks?
OpenAI's Model Spec uses a five-level chain of command to resolve conflicting instructions, with root authority at the top. The question is whether this procedural approach reliably stops lower-level inputs from overriding high-authority safeguards in practice.
companion pattern from the same document, resolving which instructions apply before this record bounds what the agent may then do
-
Can step-by-step approval miss harmful behavior patterns?
If each action an agent takes passes its individual safety check, can the overall sequence still violate system constraints? This matters because per-action inspection may miss harms that emerge only across time or composition.
Qualifies ScopeOfAutonomy: agent security is a trajectory property, so individually permissible actions can jointly violate system-level constraints
-
Can a poisoned validator still approve unsafe actions?
When a review agent reads from the same compromised memory as the retrieval agent, does it retain the authority to block unsafe actions? This tests whether a single approval point can serve as a meaningful safeguard.
Qualifies ScopeOfAutonomy's default ban: without authorization checks, a forged Validator approval lets banned high-risk actions execute every trial
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Levels of Autonomy for AI Agents
- Assistant or Actor? Student Trust, Control, and Delegation Regret When Using a General-Purpose AI Agent
- Securing Agentic AI: From Per-Action Checks to Trajectory Assurance
- Progent: Securing AI Agents with Privilege Control
- Measuring AI agent autonomy in practice
- Authenticated Delegation and Authorized AI Agents
- Code as Agent Harness
- Agents of Chaos
Original note title
OpenAI's Model Spec bounds agentic action with a ScopeOfAutonomy record requiring an ending condition and a default ban on high-risk actions