How should AI agents balance autonomy with user oversight?

Alongside their experimental finding, the authors propose "a complementary alignment target beyond the content of agent outputs: autonomy alignment, the question of whether the allocation of decision authority between agent and user matches users' expectations and supports their actual capacity for oversight." They frame this against the dominant alignment literature, which has "invested significant effort in ensuring that model output content aligns with human values," and argue decision-authority allocation is a separate thing to get right, not a side effect of getting outputs right.

The paper decomposes autonomy alignment into three levels at which it can be realized: "jointly at the model level (when the agent's policy chooses to escalate rather than act), the harness level (how escalation is enforced, as in our risk-contingent autonomy design), and the interaction level (how delegation moments are surfaced to users, as in our interfaces which highlight the risks)." In their own experiment, the model level is the agent's policy for detecting a privacy risk and choosing to escalate; the harness level is the risk-contingent autonomy mechanism that enforces the hand-back; the interaction level is the interface design that surfaces the moment of delegation to the participant. The authors situate this as consistent with "emerging views of multiple types of alignment in LLM agent systems... which extend beyond traditional output alignment to include autonomy and agency alignment."

This framework names what Can handing control back to users reduce personalization's privacy costs? measures: that experiment's risk-contingent design is one harness-level implementation of autonomy alignment, tested against two misaligned alternatives (no escalation ever, or escalation on every action). It also gives a shared vocabulary to a design conclusion reached independently in What makes people distrust AI agents they delegate to?, where students wanted "per-action autonomy settings rather than a single autonomy dial" — an interaction-level and harness-level autonomy-alignment demand, arrived at through a controlled task study rather than an experimental manipulation of autonomy itself. And it sharpens Where do user values break down in agent supervision?: values going unmet "mostly" during supervision reads, in this vocabulary, as a case where decision authority was not allocated to match what users needed to oversee effectively.

The three-level decomposition is offered as a conceptual synthesis drawn from the authors' own design choices in a single experiment, not as something they tested level-by-level; the paper does not isolate model-level, harness-level, and interaction-level misalignment from one another to show which one drives outcomes, and it does not claim the three levels are exhaustive. Its value is as a diagnostic vocabulary — a place to locate why an agent's autonomy design is failing (a policy that doesn't escalate, a harness that doesn't enforce escalation, or an interface that doesn't surface the moment) — rather than as a validated account of how the levels interact or which matters most.