Cocoa: Co-Planning and Co-Execution with AI Agents

As AI agents take on increasingly long-running tasks involving sophisticated planning and execution, there is a corresponding need for novel interaction designs that enable deeper human-agent collaboration. However, most prior works leverage human interaction to fix “autonomous” workflows that have yet to become fully autonomous or rigidly treat planning and execution as separate stages. Based on a formative study with 9 researchers using AI to support their work, we propose a design that affords greater flexibility in collaboration, so that users can 1) delegate agency to the user or agent via a collaborative plan where individual steps can be assigned; and 2) interleave planning and execution so that plans can adjust after partial execution. We introduce Cocoa, a system that takes design inspiration from computational notebooks to support complex research tasks. A lab study (n= 16) found that Cocoa enabled steerability without sacrificing ease-of-use, and a week-long field deployment (n= 7) showed how researchers collaborated with Cocoa to accomplish real-world tasks.

Introduction. Since the advent of personal computing, researchers and practitioners in artificial intelligence (AI) and human-computer interaction (HCI) have set sights on developing AI agents that can collaborate with humans to perform complex computer-based tasks [34, 61, 66, 67]. For example, paradigms such as mixed-initiative interaction envisioned fluid human-AI collaboration with user-led direct data manipulation combined with AI-led autonomous task execution [34]. Recent advancements in large language models (LLMs) and agent frameworks involving reasoning, planning, memory, and tool use [50, 90, 100, 102, 110] have accelerated the development of fully autonomous AI agents to tackle complex tasks, including shopping online [108], writing software [40, 105, 107], performing “deep research” to assemble a report based on web sources [26, 76, 83], and general computer use [2, 75, 93]. Despite the push towards full autonomy, recent work has recognized the importance of deeper human-agent collaboration for not only bolstering agent performance on complex tasks [95], but also improving accessibility [81], user control and trust [35, 69], safety [75], and user agency more generally [21], leading to the development of more collaborative human-agent systems, such as for web browsing [35, 69, 81] and general productivity [75, 94]. However, many of these systems incorporate prescribed collaborative workflows that may limit their utility and steerability. Some works use agent-guided workflows: the agent dictates what input to elicit from the user, as well as when and how the elicitation happens [35, 69, 75, 81, 94]. Users may intervene by manually interrupting or taking control of the agent [69, 75, 76], but these efforts In this work, we explore a novel instantiation of mixed-initiative interaction [34] for explicitly and flexibly delegating work in humanagent hybrid workflows by interleaving collaborative planning and execution. We conduct this exploration in the domain of scientific research, a challenging domain for human-agent collaboration due to the prevalence of knowledge-intensive workflows and nuanced decisions made with domain-specific knowledge. Existing agentic research tools primarily focus on supporting human-agent collaboration either before or after task execution. One group of systems elicits user intent and preferences before execution [26, 76, 83] (e.g., asking clarification questions about users’ goals [76], allowing plan review and modification [26, 69]). Conversely, another group helps users verify and correct the agent after it has generated an output after executing its plan [31, 73]. In practice, because planning and execution are deeply interleaved, we aim to support human-agent collaboration throughout fluid movements between the two phases. We introduce Cocoa, an interactive system that implements our approach by supporting hybrid human-agent workflows for scientific researchers in a document editor—a common site for research work. This interleaving happens via the following three ways. First, Cocoa supports human-agent co-planning: the agent, when invoked in the document, will first propose a plan of action that is fully interactive and integrated within the document. The user can then edit steps, add new ones, and assign each step to either themselves or the agent. Second, Cocoa supports human-agent co-execution: drawing inspiration from the design of computational notebooks, Cocoa allows the user and the agent to collaboratively complete plan steps, re-executing steps as desired. The user can interactively refine the agent’s intermediate outputs and also manually take over steps themselves to steer the workflow in a more desirable direction. Third, and most importantly, Cocoa interleaves co-planning and co-execution: users can smoothly transition between the two and modify their plans based on outputs from execution, and vice versa. We then evaluate Cocoa through a lab study with 16 researchers, where we found that interleaving co-planning and co-execution in Cocoa afforded greater agent steerability while maintaining ease of use compared to a strong chat baseline. We finally conducted a 7-day field deployment of Cocoa with 7 researchers, where they found the explicit delegation of work between themselves and the agent, in addition to interleaved co-planning and co-execution, to be valuable in their day-to-day research. Concretely, our work makes the following contributions:

(1) A formative study with 9 researchers that motivates the design of interactive systems that can flexibly delegate work between the user and agent across task planning and execution. (2) Cocoa, an interactive system that interleaves human-agent co-planning and co-execution in a document editor for scientific researchers.

Related work. Prior work has identified core components in LLM agent architectures to consist of memory, reasoning, planning, and tool use [50, 63, 100, 102, 110, 114]. Central to LLM agents’ longitudinal operations is long-term planning, which demands capable reasoning abilities [32] and thus is often implemented by chain-of-thought (CoT)1 [74, 85, 104, 110]. CoT provides a series of intermediate reasoning steps as exemplars in prompting to boost performance on complex reasoning tasks [74, 104]. CoT is crucial for facilitating task decomposition and therefore LLM agents’ planning capabilities [16, 85]. However, even with CoT, critical investigations of LLMs’ abilities to autonomously generate executable plans reveal limited success across diverse domains [101]. Indeed, although CoT can help improve model planning for tasks with well-defined, objective solutions that provide clear signals for reinforcement learning—e.g., numerical, tabular, and knowledge-based reasoning [50]—it may not be so effective in domains that require expert tacit knowledge to navigate ambiguous problem spaces with no single right answer [32]. Unlike model properties that show empirical improvement through scaling laws, limitations of planning in these domains may not resolve with scale alone as 1) tacit knowledge is not welldocumented in training data and is thus difficult for a model to robustly learn [17, 22, 111], and 2) there may be no “correct” workflow for CoT to follow and verify the correctness post-hoc [41]. Scientific research is one such domain [111]. In light of this, recent work has recognized the need to interactively incorporate user feedback for improving LLM agents’ planning capabilities and beyond [35, 47, 55, 69, 75, 81, 94], particularly within scientific discovery [27, 39, 68]. For example, Lawley and MacLellan [55] architected an approach where user interaction is used to guide the model in planning for unseen tasks on-the-fly using a hierarchical network of smaller actions. OpenAI’s Operator [75] allows users to “take control” of the agent to perform tasks manually. Yet, interactive techniques in this area are still nascent, despite the new challenges identified for human-agent communication [7]. Interfaces for LLM assistants (e.g., AgentGPT [87], Devin [105]) have been primarily limited to chat interfaces targeted at monitoring agent activity rather than empowering the user to proactively collaborate with the agent. An emerging body of work has started to experiment with more interactive techniques for agents. He et al. [33] separated agent planning from execution and found that user involvement during execution can significantly boost agent performance, but poses a high cognitive load. Kazemitabaar et al.

Method. 3.2.1 [F1 & F3] Project documents are promising environments for human-agent collaboration. Participants’ project documents acted as a “hub” for their projects. These documents commonly included meeting notes, planned to-do items, progress updates tied to to-do items, research questions to discuss with collaborators, and links to other documents detailing a particular project component in greater depth. These documents often served as “scratch paper” for researchers to informally leave a trace of their reasoning and high-level goals, conduct short-term and long-term planning, and progress tracking of these plans. Most documents (P1, P2–4, P7–P9) contained one or more problem statements that motivated the project and stated its core contributions. From there, participants listed subgoals and ideas for planned exploration (all participants). Participants expressed a desire for tighter integration between an AI agent and their project documents in their research workflows for two main reasons. First, they wanted to receive in-situ AI support as they plan and execute research tasks in their documents. P1 and P2 both envisioned an AI system “actively engaging with the content I’m writing [...] after I write each statement, the system could retrieve relevant papers that could provide background or related work for me to read more.” P9 appreciated “the ability to lay out the steps when I start something. [It] was very helpful [for] visualizing the outcome.” Second, participants wanted the AI agent to have access to the broader context that already existed in their project documents. When using the probe, P2 was concerned that they would need to “prompt [the chatbot] with a lot of background,” but realized they “might have [the background] written down” in their document. P3 desired closer document integration but warned that the agent may clutter their document. They suggested: “having something on the side that does not flow into where I am writing, like a side [panel]—if I want something from it, I want to bring it back into my doc.”

3.2.2 [F2] Literature search and understanding play a central role in research planning. In all project documents, we saw places where participants intended to initiate a multi-step plan (often via a how? and can we? question) to address different research tasks. The most common category of tasks are ones that were literature-augmented (e.g., literature search and understanding), which are often combined with a wide variety of other research actions, such as experiment/artifact design (P3), experiment/code execution (P4), data inspection and synthesis (P7), communicating and discussing results (P5), and ideation (P2). Literature was not only a significant component of existing plans, but was important for informing future plans. Some participants’ plans (e.g., P3, P4, P7) were formed from hypotheses they hoped to verify, which prior work may have already addressed; P7 pointed out that the hypotheses they had in mind can “actually be proven from previous papers” so they saw literature review as a planning aid. Consequently, all participants expressed a desire to use AI to help with literatureaugmented tasks—exploring and understanding relevant literature to inform decision-making. This finding echoes recent surveys on how researchers leverage LLMs to conduct research, where the most frequent usage category was information-seeking [60].

3.2.3 [F3] Participants preferred to perform higher-level reasoning and synthesis themselves. In addition to highlighting tasks where they wanted AI assistance, participants also spoke about tasks they did not want AI to automate away. In particular, participants saw higher-level reasoning and information synthesis as critical tasks they wanted to do themselves. They were also often unsatisfied with AI’s outputs on these tasks. P3 explains: “this tinkering process around reading and playing around with things is what gives you the ideas. I don’t know if I want those things automated because the process is as helpful as the final result.” P5 agreed that they “wouldn’t want it to be making the final decisions for sure. Just give me inspiration for where I can go.”

Discussion. Researchers have called for reframing the design of human-AI systems as not merely the specification of affordances, but construction of open design spaces that support human-AI co-agency [89]. That is, rather than providing a fixed configuration of human and AI agency across all interactions, users can flexibly vary the agency they wish assert and have the AI modulate its agency in response. Cocoa offers an early instantiation of this vision. We now reflect upon several key design decisions and discuss how future systems may build upon them or explore alternatives when designing for co-agency in human-agent systems. We expect that fluid re-planning will continue to be important and challenging to design for in future systems. Plans are bound to shift as the user reviews agent outputs and becomes inspired with new ideas. For example, researchers in both our lab and deployment studies dynamically re-planned with the agent upon gaining more clarity on what they actually wanted to explore after co-executing earlier steps (see Sections 6.2 and 7.2.3). In Cocoa, re-planning replaces outdated steps with new ones. However, participants sometimes expressed a desire to preserve the old steps because their outputs can still be useful to reference in the future. We thus see plan branching as a promising design pattern for exploring counterfactual plan paths. For example, systems can allow users to swap certain steps in and out of the plan, or set checkpoints at specific steps to anchor exploration. By doing so, useful work can be better retained while supporting flexible plan redirection. Relatedly, the length of the plan should be calibrated depending on the task. Some tasks, such as open-ended brainstorming, benefit from shorter, partial plans where next steps can be flexibly determined based on prior execution. Indeed, a few participants expressed a preference for a step-by-step planning approach over Cocoa’s default plans when conducting brainstorming, where they did not know what next step would be until they saw the results of the previous step. On the other hand, for tasks with more structured and predictable workflows, such as literature review, participants appreciated interacting with a full plan. We calibrated the lengths of plans generated by Cocoa based on participants’ plans in our formative study. We recommend designers of human-agent systems to consider what the appropriate plan length is for tasks they hope to support, and calibrate accordingly. Finally, user expertise and upskilling potential should be considered when designing for an appropriate level of agent control. As surfaced in our formative study (Section 3.2.3), there are two primary reasons why users prefer keeping a task for themselves over delegating it to the agent. First, the user can perform the task more effectively than the agent. This is often the case where the user holds more expertise (e.g., brainstorming research questions in a user’s primary research area). Second, the task is intrinsically valuable for the user to do themselves, even if an agent is capable of automating it (e.g., reading seminal papers to develop research taste). If a human-agent system is designed for expert users who want to regularly hone their skills and intuition, providing many levels of control throughout agent planning and execution (as in Cocoa) may be appropriate. If the system is for navigating unfamiliar domains with agents and users are not looking to upskill, fewer controls may enhance user experience. Because interactions such as co-planning and co-execution can burden the user with additional cognitive load, determining the user’s capacity and willingness to take on more cognitive load can inform the appropriate level of control to afford.

Conclusion. In this paper, we presented Cocoa, an interactive system for scientific researchers to flexibly collaborate with an AI agent in a document editing environment to tackle open questions and tasks within their research projects. Motivated by a formative study with 9 researchers, Cocoa introduced a new approach for human-agent collaboration—interleaved co-planning and co-execution—by which a human and agent can iteratively create plans of action and execute the plan in a mixed-initiative manner that centers human agency. Our lab (n= 16) and field deployment (n= 7) studies showed that participants benefited from interleaved co-planning and coexecution in the lab and in the field. Compared to a strong baseline with a more familiar chat interface, Cocoa enhanced agent steerability without sacrificing ease of use. In the lab study, participants’ main mode of co-execution was refining agent outputs, but when Cocoa was integrated into their research workflows in the field over a 7-day period, participants took advantage of user steps to guide the agent with their expertise and exert more of their agency. Overall, our work demonstrates the promise of our approach for flexibly mediating human and AI agency. Importantly, our work offers many practical takeaways for the design of human-agent systems, such as leveraging user steps and planning ahead for agent cost management and oversight. AI agents have exciting potential to transform our digital experiences and advance long-standing visions held by HCI and AI communities. However, these transformations and advancements also demand renewed attention to strategies for ensuring effective and flexible human-agent collaboration. Our work advances efforts in this emerging frontier.