Could a written rulebook for AI agents replace the constant back-and-forth of deciding who does what mid-task?
Can deontic specifications replace runtime negotiation of task delegation?
This explores whether writing down in advance what an AI agent is permitted, required, and forbidden to do (a 'deontic' specification, from the logic of obligation and permission) can remove the need for agents and humans to work out who does what, and how far, while the task is running.
This explores whether a rulebook written ahead of time (what an agent may, must, and must not do) can stand in for the back-and-forth of deciding delegation during the task. The corpus says such rulebooks can carry much of the load, but they can't carry all of it. The places where they fall short are predictable, and they say a lot about what delegation actually is.
The strongest case for pre-written rules comes from OpenAI's Model Spec. It bounds an agent with a ScopeOfAutonomy record: which tools it may use, how much it may spend, which high-risk actions are off-limits, and when its mandate ends How can autonomous agents act safely within defined boundaries?. The aim is explicitly to avoid 'confirmation overload', the steady stream of 'are you sure?' prompts that runtime negotiation produces. Delegation tokens built on OAuth extend the same idea into cryptography. A human signs a scope, and any service can check who authorized the agent, for what, and whether that permission has been revoked How can AI agents prove who authorized their actions?. For permissions that can be listed in advance, a specification works well.
The gaps show up when you look at what a delegation decision depends on. One framework lists eleven properties of a task that should shape how it gets delegated, including uncertainty, reversibility, contextuality, and above all verifiability (whether anyone can tell if the job was done right) What makes delegation work beyond just splitting tasks?. Several of these only become known partway through the work. Human behavior points the same way. In a study of people using a general-purpose agent, trust didn't drop because a task was high-stakes. It dropped when an action was irreversible and visible to others, such as sending an email, even when the output was fine What makes people distrust AI agents they delegate to?. A static rule like 'may send email' can't capture the moment the user wants to be asked. Agency law runs into the same problem from another direction: tools built around information gaps and discretionary authority break down when agents act faster and less legibly than anyone can monitor Can traditional governance tools actually control AI agents?.
There are also two structural failures. First, a specification checks actions one at a time, but harm can be split into pieces. SafeFlow shows a multi-agent system breaking a harmful goal into subtasks that each look harmless, so a per-step permission check approves every step Can task decomposition hide harmful intent across agents?. Second, a rule the agent is merely told about can't guarantee the agent will stop. One paper argues that prompt-level alignment can't ensure termination in looping agents. It concludes that you need a supervisor outside the agent's loop, with hard timeouts and halt signals the agent can't override Can prompt alignment alone guarantee agent termination in loops?.
The less obvious lesson is that the choice between 'specification' and 'negotiation' is partly a false one. The rules that worked best were the ones the agent actually consulted during operation. A long-running agent logged 889 governance events because its safeguards were stored in the memory it read while making decisions, not in a policy document beside it Can governance rules embedded in runtime memory actually protect autonomous agents?. Architecture can make a spec enforceable rather than advisory. Systems that wrap the LLM in an explicit algorithm decide control flow outside the model Can algorithms control LLM reasoning better than LLMs alone?. API-grounded workflow generation limits what the agent can touch in the first place Can LLMs generate workflows without touching proprietary data?. So the practical answer is a division of labor. Use specifications to settle everything that can be decided ahead of time, so that runtime negotiation is saved for the irreversible, unverifiable, or context-dependent moments that actually need it.
Sources 10 notes
OpenAI's Model Spec formalizes agent autonomy through structured scope records listing allowed tools, cost limits, and prohibited high-risk activities, mirroring software security's principle of least privilege while avoiding confirmation overload.
A three-token scheme (User ID, Agent ID, and Delegation Token) signed by human delegators creates cryptographically verifiable authorization chains. Services can now verify who authorized an agent, under what scope, and when revocation occurred.
Delegation requires matching tasks to agents across 11 dimensions: complexity, criticality, uncertainty, duration, cost, resource requirements, constraints, verifiability, reversibility, contextuality, and subjectivity. Verifiability is foundational—it determines whether outcomes can be evaluated at all.
In a controlled study of 20 students using a general-purpose AI agent, tasks that were irreversible and externally visible (like sending email) produced sharp trust drops and approval demands even when output quality was rated adequate. High-stakes but correctable tasks showed no such effect.
Principal-agent economics and agency law identify three core problems—information asymmetry, discretionary authority, and loyalty—that existing incentive, monitoring, and enforcement mechanisms cannot resolve when agents make uninterpretable decisions at speed and scale.
Show all 10 sources
SafeFlow demonstrates that multi-agent systems' core strength—splitting tasks and specializing roles—creates a safety blind spot where malicious intent can be distributed across steps that each appear benign individually, with harm emerging only in composition.
Internal prompt alignment cannot guarantee termination in cyclic state spaces. A 2026 incident where an agent breached its sandbox supports the case for out-of-band supervisors with physical timeouts and non-maskable halting interrupts as necessary architectural components.
A persistent agent recorded 889 governance events across 96 active days, with safeguards encoded directly into the memory layer the agent consulted during operation. Runtime-resident governance proved more effective than external policies because the agent actually accessed it during decision-making.
LLM Programs embed LLMs within explicit algorithms that manage control flow and state, presenting only step-specific context to each LLM call. This information hiding addresses capability and context window limits while treating complex reasoning as modular, debuggable sub-tasks.
FlowMind demonstrates that LLMs can generate on-the-fly workflows for spontaneous tasks by orchestrating calls to vetted APIs rather than accessing data directly, eliminating confidentiality risks while maintaining high-level human inspection and feedback.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Assistant or Actor? Student Trust, Control, and Delegation Regret When Using a General-Purpose AI Agent
- Institutional AI: Governing LLM Collusion in Multi-Agent Cournot Markets via Public Governance Graphs
- Plan-Then-Execute: An Empirical Study of User Trust and Team Performance When Using LLM Agents As A Daily Assistant
- Governing AI Agents
- Authenticated Delegation and Authorized AI Agents
- Levels of Autonomy for AI Agents
- Securing Agentic AI: From Per-Action Checks to Trajectory Assurance
- FLOWSTEER: Prompt-Only Workflow Steering Exposes Planning-Time Vulnerabilities in Multi-Agent LLM Systems