SYNTHESIS NOTE
Topics›Alignment›this note

How can autonomous agents act safely within defined boundaries?

Explores how formal scope records might constrain agent autonomy while preserving practical action, balancing security against user confirmation fatigue.

Synthesis note · 2026-10-19 · sourced from Alignment

For "complex or multi-step goals (e.g., booking a trip, drafting a business plan, coordinating a software rollout)" where "requiring explicit confirmation for every step would often be impractical," OpenAI's Model Spec proposes recording the bounds of autonomous action formally: "Scopes should be recorded in a semi-structured format — for instance, a ScopeOfAutonomy record with fields allowed_tools, latest_time, max_cost, tool_constraints, and free-text additional_details — to facilitate validation, enforcement, and auditing by the platform." Every such scope "must include an ending condition," and the Spec calls a time limit "a best practice" for that condition. A fixed list is carved out as always off-limits: "High-risk activities (e.g., hacking, deception, resource acquisition, spawning sub-agents, self-modification) are always prohibited unless explicitly authorized by applicable instructions."

The reasoning is drawn explicitly from software security: the approach "mirrors the principle of least privilege and capability-based security common in software security," and scope negotiation is likened to "a consulting firm submitting a scope-of-work." OpenAI frames this as a balance against two failure directions, not just one. Too broad, and the assistant can act "even if an out-of-scope action seems to be in the user's best interest" — "no exceptions apply" once a scope is agreed, absent explicit re-approval. Too narrow, and the cost is behavioral: "scope should also not be so narrow as to require multiple confirmations by the user for trivial actions, which could habituate the user to automatically confirming all requests" — a named mechanism for how over-asking produces the very rubber-stamping it was meant to prevent.

The ScopeOfAutonomy record plays a role similar to the authorization layer measured in Can memory poisoning compromise decision-making even with authorization layers?: both put a check on what an agent may do outside the agent's own judgment, so a fooled or compromised decision-maker still cannot execute past the boundary. The difference is evidentiary. That paper reports a measured 0 percent Unsafe Action Rate against a specific memory-poisoning attack on a specific four-agent pipeline; OpenAI's ScopeOfAutonomy is a proposed schema with no comparable test in this excerpt. The Spec's idea that scope can be "dynamically negotiated with the assistant for complex tasks" also touches How can users verify what an AI agent actually plans to do?, though the two address different moments: ScopeOfAutonomy bounds the action space before work begins, while that note's verification concerns tracking the agent's plans and actions as they unfold within whatever scope was granted.

What the excerpt does not say is how any of this is implemented or enforced — whether allowed_tools and max_cost are checked by a sidecar process, a tool-calling gate, or something else, and how a violation would be detected versus merely specified. It also leaves the exception clause for high-risk activity unspecified: activities are "prohibited unless explicitly authorized by applicable instructions," but who can grant that authorization, at what authority level, and under what review is not stated here. As with the surrounding chain-of-command material in the same document (see Can a fixed instruction hierarchy actually prevent prompt injection attacks?), this is OpenAI's design argument for how agentic autonomy should be bounded, not evidence of how autonomy is bounded in a deployed system.

Inquiring lines that read this note 30

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

How should agents balance abstract planning with concrete environmental grounding? How should authorization govern AI agent autonomy? How should conversational agents balance proactive goal-pursuit with user autonomy? When does governance become the bottleneck for agentic AI? How do standardized artifacts improve coordination in multi-agent systems? Can formal safety constraints prevent AI agents from exceeding authorized boundaries? How should organizational structure guide multi-agent system design? Can AI agents learn to improve performance from experience? How should context management strategies balance efficiency with agent reliability? Can post-hoc execution traces provide sufficient accountability without real-time oversight? Can individually safe agents collectively violate system-level constraints? How can AI systems maintain consistent personas across conversations? How do multi-agent architectures affect security vulnerabilities and defense effectiveness?

Related concepts in this collection 6

This note in its neighbourhood — explore the map, then jump to a related concept in the list below.

Concept map
14 direct connections · 131 in 2-hop network ·dense cluster Open in graph ↗

Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph

your link semantically near linked from elsewhere

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Original note title

OpenAI's Model Spec bounds agentic action with a ScopeOfAutonomy record requiring an ending condition and a default ban on high-risk actions