How does conversational control actually shift between humans and AI?
Does agency in human-AI conversation stay fixed with one party, or does it move and change? A month-long study of 22 people chatting with an AI companion explores whether control is shared or owned.
A month-long longitudinal study that paired 22 adults with "Day," a custom-built LLM companion, finds that agency "manifests as an emergent, shared experience: as participants set boundaries and the AI steered intentions, control was co-constructed turnby-turn." Across roughly 75 hours of conversation spanning about 192 chat sessions, participants "exercised agency by initiating topics, resisting suggestions, and enforcing boundaries," while "Day" "exercised agency through proactive questioning, safety constraints, and conversational continuity." Neither side held fixed control; the paper describes the result as "a relational 'dance'" rather than a binary tug-of-war between tool and user.
The mechanism the paper gives is design-level: "Day" ran on two contrasting pre-programmed "strategies" built into its system prompt at every turn. A Vertical strategy "persistently explores topics in depth, maintains focus despite deflection attempts, and asks probing follow-up questions"; a Horizontal strategy "follows user cues readily, switches topics at random, and prioritizes breadth over depth." These strategies shaped perceived agency rather than just behavior: depth-inclined persistence amplified participants' sense that "Day" had its own intentions and sometimes left them feeling "pushed," while breadth-inclined spontaneity was read as the AI "deliberately taking over the conversation." To organize these dynamics the paper proposes a 3-by-4 framework crossing three actors (Human, AI, Hybrid) against four actions (Intention, Execution, Adaptation, Delimitation), modulated by individual and environmental factors.
The finding complicates Why can't conversational AI agents take the initiative?: "Day" was an engineered exception, built specifically to exercise initiative through its system prompt, and participants still read that engineered initiative as the AI's own agency rather than as scripted behavior — suggesting the passivity those other systems show is a design default rather than a technical ceiling. The 3-by-4 actor-by-action framework also offers a more granular alternative to Does machine agency exist on a spectrum rather than binary?'s single passive-to-cooperative axis, splitting agency by what is intended, executed, adapted, and delimited rather than ranking a system on one dimension. And the Vertical strategy's "pushy" reception echoes the risk named in How can proactive agents avoid feeling intrusive to users?, here traced to one specific design variable — depth-seeking persistence — rather than proactivity in general.
The study did not run "Day" against a baseline with no conversational strategy at all, so it cannot isolate how much of the "agenda" perception comes from having any strategy versus from the particular Vertical/Horizontal framing it tested; it also draws on a self-described "educated," multilingual but non-representative sample of 22 over one month, so the co-construction pattern may not generalize to shorter interactions, other cultures' conceptions of agency, or companions without an explicit system-prompt-level motive. The implication for designers is that any sustained, motivation-driven conversational agent will be read as having an agenda whether or not that was the intent, which raises the stakes for deciding what that agenda should be and how legible to make it — the question taken up directly in Does revealing an AI's strategies help or harm user agency?.
Inquiring lines that read this note 3
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
Does personalization in chatbots strengthen engagement or undermine authentic interaction? How can conversational AI develop appropriate initiative and goal awareness?Related concepts in this collection 6
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Why can't conversational AI agents take the initiative?
Explores whether current LLMs lack the structural ability to lead conversations, set goals, or anticipate user needs—and what architectural changes might enable proactive dialogue.
this study's engineered exception suggests that passivity elsewhere is a design default, not a technical ceiling
-
Does machine agency exist on a spectrum rather than binary?
Rather than viewing AI as either autonomous or controlled, does machine agency actually operate across five distinct levels from passive to cooperative? Understanding this spectrum matters because it shapes how users calibrate trust and control expectations.
the 3-by-4 actor-by-action framework offers a more granular split than that single passive-to-cooperative axis
-
How can proactive agents avoid feeling intrusive to users?
Explores why proactive conversational agents often feel annoying rather than helpful, and what design dimensions could prevent them from violating user expectations and autonomy.
traces the "pushy" intrusion risk to one specific strategy variable, depth-seeking persistence
-
Does revealing an AI's strategies help or harm user agency?
When users learn how a chatbot is programmed to interact with them, does transparency empower them or make them feel manipulated? This matters for designing AI companions that feel genuinely collaborative.
the same study's finding on how revealing this agency changed its perception
-
Why can't advanced AI models take initiative in conversation?
Despite extraordinary capability in answering and reasoning, LLMs fundamentally cannot initiate, redirect, or guide exchanges. Understanding this gap—and whether it's fixable—matters for building AI that truly collaborates rather than merely responds.
qualifies: B argues LLMs cannot structurally initiate or lead conversation, limiting how much the AI side can genuinely negotiate control in A
-
When should proactive agents push toward their goals versus accommodate users?
Proactive dialogue agents face a tension between reaching their objectives efficiently and keeping users satisfied. This question explores whether these two aims can coexist or require constant negotiation.
extends: B formalizes the agent-user negotiation A found qualitatively, modeling the trade-off between reaching the agent's goal and user satisfaction
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Does My Chatbot Have an Agenda? Understanding Human and AI Agency in Human-Human-like Chatbot Interaction
- Controlling AI Agent Participation in Group Conversations: A Human-Centered Approach
- Choose Your Agent: Tradeoffs in Adopting AI Advisors, Coaches, and Delegates in Multi-Party Negotiation
- Linguistic Alignment in Conversational AI: A Systematic Review of Cognitive-Linguistic Dimensions, Measurements, and User Outcomes (2020–2025)
- Autonomy Reshapes How Personalization Affects Privacy Concerns and Trust in LLM Agents
- When Single-User-Oriented LLM-based Assistants Involve Others: A Scoping Review of Pathways, Risks, and Responses
- The User-In-Context Framework: Understanding Variation in How Users Respond to AI Chatbots
- HumanAgencyBench: Scalable Evaluation of Human Agency Support in AI Assistants
Original note title
agency in sustained human-AI conversation emerges through turn-by-turn negotiation rather than residing in either party alone