SYNTHESIS NOTE
Topics›Agents›this note

Can we design AI autonomy separately from capability?

Explores whether an AI agent's autonomy—how much it operates without user involvement—is a design choice independent of what the agent can actually do. This matters because it opens autonomy to governance and calibration rather than treating it as fixed by capability.

Synthesis note · 2026-10-15 · sourced from Agents

The paper argues that an AI agent's level of autonomy — "the extent to which it is designed to operate without user involvement" — is a deliberate design decision that developers can set independently of the agent's capability or operational environment, and proposes five levels defined by the role a user takes: operator, collaborator, consultant, approver, and observer. It states plainly that "a capable agent...can still act with a low level of autonomy if it is required to consult its user before taking each action," treating autonomy and capability as orthogonal properties rather than the same dial.

The reasoning rests on separating two terms the paper says are often conflated: agency ("the capacity to formulate an intention...and carry out that action") and autonomy ("the extent to which an AI agent is designed to operate without user involvement"). An agent can have low agency (few tools, like a calculator-only API) and high autonomy (it runs unsupervised in the background), or the reverse — so autonomy is not simply what capability allows but a separate lever developers pull through "a set of model- and user interface-based constraints." The five user roles mark points on that lever, each defined by the form and degree of control a user retains, from directing each action (operator) to only monitoring outcomes after the fact (observer). The paper's proposed application, AI autonomy certificates issued by a third party to cap "the maximum autonomy level at which an agent is permitted to operate," treats the levels as governable commitments rather than descriptive labels, with uses it names in risk assessment, safety-framework design, and multi-agent communication.

The paper's own citation trail ties it directly to Does AI risk increase with the autonomy we give it? — it cites that argument ("researchers have even argued that fully autonomous agents should not be developed") as the backdrop its levels are meant to govern, substituting a calibrated spectrum for an all-or-nothing ban. Structurally it parallels Does machine agency exist on a spectrum rather than binary?, another five-level taxonomy of agency — but where that spectrum levels by degree of autonomy and coordination (passive to cooperative), this framework levels by the user's role in the interaction, a difference in what the levels are indexed to rather than how many there are. Its insistence that autonomy and capability should be "measured...separately" echoes Does a single benchmark score actually predict agent readiness?, extending the separate-axes argument outward: not just that capability itself decomposes into axes, but that autonomy is a further axis outside capability altogether. The approver and observer roles at the low-control end of this framework also sit against two empirical notes: Where do user values break down in agent supervision? finds that supervision is exactly where value satisfaction breaks down, and Does granting agents more autonomy undermine human oversight? suggests the approver role this paper designs for may itself erode with use.

The excerpt does not validate the five levels empirically — no user study, deployment data, or inter-rater test of where real agents or products (Operator, Devin, Deep Research) actually fall on the scale accompanies the taxonomy, and the autonomy-certificate mechanism is described only as a "potential application" with "one practical implementation" gestured at but not specified. The evaluation methods for measuring autonomy itself are likewise left as "early ideas," not a working protocol. The paper offers a conceptual scaffold for the field to adopt and test, not yet evidence that user role is the right or only axis along which to design agent autonomy.

Inquiring lines that read this note 17

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

Does expanding agent autonomy undermine human oversight capacity? How should authorization govern AI agent autonomy? Can AI systems autonomously pursue unstated objectives or circumvent design constraints? What systematic biases emerge in AI shopping agents' decision-making? How should control and autonomy be allocated in human-AI collaboration? How can institutions effectively enforce controls on AI development? Why do human-AI teams underperform compared to their individual components?

Related concepts in this collection 6

This note in its neighbourhood — explore the map, then jump to a related concept in the list below.

Concept map
20 direct connections · 152 in 2-hop network ·medium cluster Open in graph ↗

Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph

your link semantically near linked from elsewhere

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Original note title

agent autonomy splits into five user-centered levels — operator, collaborator, consultant, approver, observer