Can an AI assistant get smarter about helping you over time without quietly starting to make your calls for you?
How can agents preserve user autonomy while providing assistance over time?
This explores how an AI agent can keep helping someone over many tasks and sessions, and learn along the way, without slowly taking decisions away from them.
This explores how an agent can keep helping someone over many tasks and sessions, and learn as it goes, without slowly taking decisions away from them. The corpus doesn't treat autonomy as one setting you pick once. It treats it as something you design into several places: how much the agent is allowed to do, when it stops to check in, what it remembers, and who it is actually working for.
The simplest version is a dial. Designers are settling on a per-task autonomy slider with three positions: suggest-only, draft, and execute. That lets a user give an agent full control over one kind of task while keeping it at suggest-only on another Should agent autonomy be a continuous slider or fixed levels?. The note points out that this is shared vocabulary among designers, not a tested result. OpenAI's Model Spec takes a more formal route. It writes autonomy down as a scope record: which tools the agent may use, how much it may spend, which risky actions are off-limits, and when its permission ends How can autonomous agents act safely within defined boundaries?. The ending condition matters for long-running help. A grant of permission that expires can't quietly turn into permanent control. This also fixes a real annoyance. If an agent asks for confirmation on everything, people start approving without reading, and their autonomy becomes a formality.
A second finding is that the best control points are often in the middle of a task, not at the start. In Cocoa, users can hand individual plan steps to themselves or to the agent, and they can change the plan while it runs. People steered the agent better than they did through chat, and it was no harder to use Can users steer AI agents better by revising plans mid-execution?. Magentic-UI starts from a frank admission: nobody knows when an agent should hand a decision back to a human. So instead of trying to time that one handoff perfectly, it spreads oversight across six touchpoints: planning together, doing tasks together, guards on risky actions, checking results, memory, and running several tasks at once When should human-agent systems ask for human help?. The lesson is that autonomy holds up better with many small chances to step in than with one big approval at the start.
The "over time" part of the question brings in something less obvious: what the agent learns. Several systems now improve without retraining the model itself. They store experiences, skills, or summaries in a memory the agent reads from Can agents learn continuously from experience without updating weights?, Can agents learn new skills without forgetting old ones?, Can agents compress their own memory without losing critical details?, or they mix quick fixes after failures with slower updates during idle time Can agents adapt without pausing service to users?. None of these papers is about autonomy. But learning stored as a skill library or a set of saved cases can, in principle, be opened, read, and edited by the user. Learning baked into the model's weights can't. Magentic-UI already lists memory as one of its oversight touchpoints. An agent whose habits live somewhere you can inspect is one you can still correct a year later.
The hardest problem is loyalty. Agency law and economics name three long-standing problems with anyone who acts on your behalf: they know things you don't, they have room to make their own calls, and their loyalty may be split. Traditional fixes like monitoring and incentives break down when an agent makes decisions too fast and too opaquely to audit Can traditional governance tools actually control AI agents?. Another line of work argues that "serve one user faithfully" isn't even enough as a goal once agents operate at scale. It calls for balancing many people's interests, making agent behavior visible, and holding developers and deployers accountable for harm Should AI agents serve one user or balance many interests?. Put together: sliders and checkpoints protect your control over what the agent does, but not over whose interests shape its suggestions. The corpus has a lot on interface controls and continual learning. It has little that directly tests whether users keep real autonomy over months of use, so that question is still open.
Sources 10 notes
The AI UX Playground glossary names the autonomy slider as a continuous control letting users dial agent independence per task, collapsing agency levels into three adjustable points: suggest-only, draft, execute. The pattern shows shared vocabulary among designers, though validation against other control shapes remains absent.
OpenAI's Model Spec formalizes agent autonomy through structured scope records listing allowed tools, cost limits, and prohibited high-risk activities, mirroring software security's principle of least privilege while avoiding confirmation overload.
Cocoa's system, which lets users assign plan steps to themselves or an agent and revise plans mid-execution, afforded greater steerability than chat-baseline interaction in a lab study (n=16) while maintaining ease of use, with field users valuing explicit work delegation.
Magentic-UI identifies co-planning, co-tasking, action guards, verification, memory, and multitasking as mechanisms that work around the lack of ground truth for optimal deferral timing. Rather than solving the timing problem directly, these mechanisms distribute decision-making across multiple touchpoints.
AgentFly formalizes agent learning as a Memory-augmented MDP with three memory modules (case, subtask, tool) that enable credit assignment and policy improvement entirely through memory operations. The approach achieved 87.88% on GAIA validation without modifying LLM parameters.
Show all 10 sources
VOYAGER demonstrates that storing executable skills in an embedding-indexed library and composing complex skills from simpler ones allows agents to learn continuously while avoiding the forgetting that occurs with weight-update-based methods. Environmental feedback refines skills while an automatic curriculum drives continual exploration.
DeepAgent's autonomous memory folding consolidates interaction history into episodic, working, and tool memory schemas. This reduces token overhead while letting agents pause to reconsider strategies—the autonomy and structure together avoid degradation that plagues poorly designed consolidation.
MetaClaw demonstrates that deployed agents require both rapid skill injection from failures (seconds, zero downtime) and slower gradient-based optimization during idle windows (minutes to hours). The two mechanisms reinforce each other, with better policies producing more informative failures and richer skills enabling higher-reward trajectories.
Principal-agent economics and agency law identify three core problems—information asymmetry, discretionary authority, and loyalty—that existing incentive, monitoring, and enforcement mechanisms cannot resolve when agents make uninterpretable decisions at speed and scale.
Research argues the traditional single-agent-single-user alignment framing is insufficient at scale. Instead, governance should rest on inclusivity (balancing diverse interests), visibility (rigorous tracking and transparency), and liability (developer and deployer accountability for harms).
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Levels of Autonomy for AI Agents
- HumanAgencyBench: Scalable Evaluation of Human Agency Support in AI Assistants
- Governing AI Agents
- Useful Memories Become Faulty When Continuously Updated by LLMs
- AgentFly: Fine-tuning LLM Agents without Fine-tuning LLMs
- MetaClaw: Just Talk — An Agent That Meta-Learns and Evolves in the Wild
- SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning
- Continual Learning Bench: Evaluating Frontier AI Systems in Real-World Stateful Environments