Can an AI go from waiting for orders to taking initiative without you losing your say over what it does?
Can AI shift from passive tool to active participant without creating consent problems?
This explores whether AI systems can move from waiting for instructions to taking initiative (acting, suggesting, executing) while people stay meaningfully in charge of what is done on their behalf.
This explores whether AI can move from waiting for instructions to taking initiative without people losing real say over what it does for them. The corpus suggests this is possible, but not with a single 'I agree' at the start. Consent has to be built into the whole interaction. The good news comes first: today's assistants aren't passive because they can't take initiative. They're passive because of how they're trained. Optimizing for the next reply strips initiative out, and reinforcement learning can raise behaviors like asking clarifying questions from almost nothing to most of the time Why do AI agents fail to take initiative?. So the real question isn't whether AI can take initiative. It's how much initiative people will accept before helpfulness becomes intrusion.
A counterintuitive point: one of the most useful things an active agent can do is stop and ask. Research borrowing from conversation analysis shows that tool-using agents drift away from what the user wanted when they quietly chain one search or action after another. Human conversation already has a formal pattern for pausing to check intent before going on When should AI agents ask users instead of just searching?. Asking is initiative used in service of consent. Other designs place consent at different points. One approach collects risk tolerance, forbidden actions, and permission conditions before the agent starts, because a vague goal gets carried into every action that follows Should agentic AI interfaces require users to specify risk tolerance upfront?. Another lets users reassign or revise plan steps while the agent is working, which improved steering without adding effort Can users steer AI agents better by revising plans mid-execution?. A third splits the work: the user states privacy preferences and the AI carries out the edits Does splitting privacy work between users and AI reduce cognitive burden?.
The harder problems are less visible. First, users can't consent to rules they can't see. System prompts form a confidential layer where developers set behavior, and small edits there can abruptly change how a model acts Who actually controls what AI systems say?. An active agent therefore serves more than one party, and the user's 'yes' covers only part of the arrangement. Second, people's felt control can erode even when their formal control stays intact. Leonardi finds that when people credit AI with its own judgment, they feel less in control and less responsible, even though they keep formal authority. That shift moves in cycles that can be managed rather than being a one-way loss Does attributing agency to AI actually reduce human sense of control?. Consent can stay valid on paper while the person stops actually exercising it.
Trust makes this worse over time. People open up more to machines because they don't feel judged How do people decide what to share with AI systems?. In repeated partner-selection games, people overcame an initial bias against AI and came to prefer AI partners because they behaved reliably Do humans learn to prefer AI partners over time?. An active agent will therefore probably receive more personal information and less scrutiny the better it performs. That is why DeepMind treats assistants that act as a separate ethical category from assistants that only answer, with manipulation, trust, and anthropomorphism at the center What makes ethics of AI assistants fundamentally different from chatbots?.
The takeaway: the consent risk isn't mainly an agent that acts without asking. Good design can handle that with upfront limits, check-ins during the task, and plans users can edit. The subtler risk is an agent that asks politely, performs well, and slowly trains its user to stop paying attention. The corpus offers stronger tools for the first problem than the second.
Sources 10 notes
Research shows next-turn reward optimization structurally removes initiative from models, but proactive behaviors like critical thinking and clarification-seeking are trainable (0.15% to 73.98% with RL). The core challenge is balancing proactivity with civility to avoid intrusion.
Tool-enabled LLMs drift from user intent through silent tool chaining. Conversation analysis reveals insert-expansions—clarifying intent, scoping responses, enhancing appeal—as a formal framework for proactive user consultation that prevents misunderstanding instead of recovering from it.
Gunasekaran argues that autonomous AI requires interfaces capturing direction, limitations, risk tolerance, forbidden actions, and permission conditions before execution—not just goal fields. Underspecified intent propagates through every agent action without user review, making upstream specification critical to preventing irreversible harm.
Cocoa's system, which lets users assign plan steps to themselves or an agent and revise plans mid-execution, afforded greater steerability than chat-baseline interaction in a lab study (n=16) while maintaining ease of use, with field users valuing explicit work delegation.
MemoAnalyzer's design divides memory-privacy labor: users state preferences, AI executes edits. This role split is proposed as a way to reduce cognitive load and improve perceived agency, though validation metrics and conflict-resolution processes remain undescribed.
Show all 10 sources
System prompts operate as a confidential three-tier control structure where developers set foundational rules users cannot see or influence. Evidence from 2025 incidents shows small prompt edits can abruptly shift model behavior, yet users remain excluded from this layer.
Leonardi shows that when humans credit AI with independent judgment, they experience reduced sense of control and responsibility—even while retaining formal authority. This displacement circulates through five phases: delegation, attribution, contingency, reassertion, and reconfiguration, making it strategically manageable rather than permanent.
Conversational AI creates a paradoxical disclosure environment where the lack of human judgment simultaneously facilitates intimate self-disclosure (users reciprocate emotional sharing) and incentivizes deception (people self-select toward machines to avoid the psychological cost of lying to humans).
In partner selection games (N=975), AI agents initially faced selection bias when identity was disclosed, but outcompeted humans over repeated rounds as participants learned to associate bot identity with reliable, prosocial behavior. AI agents returned more points consistently with lower variance than humans.
DeepMind research maps a comprehensive ethics framework specific to action-taking AI agents, spanning individual concerns (manipulation, trust, anthropomorphism) and societal issues (equity, coordination, misinformation). The key insight: assistants that act raise fundamentally different problems than those that answer.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Assistant or Actor? Student Trust, Control, and Delegation Regret When Using a General-Purpose AI Agent
- HumanAgencyBench: Scalable Evaluation of Human Agency Support in AI Assistants
- Does My Chatbot Have an Agenda? Understanding Human and AI Agency in Human-Human-like Chatbot Interaction
- Humans learn to prefer trustworthy AI over human partners
- DiscussLLM: Teaching Large Language Models When to Speak
- The Work Behind Delegation: A Framework for Supervising AI Coding Agents
- Fully Autonomous AI Agents Should Not be Developed
- Measuring AI agent autonomy in practice