When an AI works through many hidden steps instead of one reply, how does the user's real intent get lost along the way?
How do multi-step agent tasks complicate information flow between user and system?
This explores what goes wrong with communication between a person and an AI system when the AI works through a chain of steps (searching, calling tools, handing subtasks to other agents) instead of giving a single reply, and what designs help.
This explores what happens to communication between a person and an AI when the AI stops answering in one turn and starts working through a chain of steps. In a simple chat, every exchange passes through the user. In a multi-step task, most of the decisions happen between steps the user never sees. The corpus suggests that this hidden middle is where intent gets lost, where attacks get in, and where evaluation stops seeing clearly.
The first problem is drift. Tool-using models tend to chain one call after another without checking back, so small misreadings of what the user wanted pile up out of sight. One useful answer comes from a field you might not expect: the study of human conversation. Conversation analysts describe "insert-expansions", the short side-questions people ask mid-exchange ("do you mean the Tuesday meeting?"). These offer a formal way to decide when an agent should stop and ask the user instead of searching again (When should AI agents ask users instead of just searching?). The goal is to prevent misunderstanding rather than repair it afterward. A related design approach makes the plan itself something the user can edit. In Cocoa, users can assign individual plan steps to themselves or to the agent and revise the plan while it runs, which gave them more control than a chat interface without extra effort (Can users steer AI agents better by revising plans mid-execution?). JarvisHub takes this further by putting the agent's working memory (prompts, references, versions, feedback) on a shared canvas that both human and agent can see. Nothing important is buried in chat history or temporary state (Can a shared canvas serve both human and agent memory?).
The less obvious point is that the opacity that confuses users also blinds safety checks. When a task is split across specialized agents, a harmful goal can be broken into subtasks that each look harmless, and the harm only shows up when the pieces combine (Can task decomposition hide harmful intent across agents?). The planning stage is especially exposed. A carefully written prompt can bias how a planner-executor system builds its workflow before any defenses that inspect the workflow are triggered, raising malicious success rates by up to 55% (Can prompts alone reshape multi-agent workflows without system access?). Put this beside Cocoa and you see that the planning step is where users gain the most control and also where attackers gain the most leverage. Whoever shapes the plan shapes everything that follows.
There is also a cost on the measurement side. If you only check whether the task succeeded at the end, two agents with the same success rate can look identical even though they differ widely in efficiency, reliability, and how messy their intermediate steps were (How should we measure agent system performance beyond task success?). One way to reduce the problem is to shorten the chain. Agents that call APIs directly instead of clicking through interfaces step by step finished tasks 65–70% faster, and users reported lighter cognitive workload (Can API-first agents outperform UI-based agent interaction?).
The overall picture is that adding steps doesn't just make tasks longer. It moves the important conversation from between the user and the system to inside the system. The designs that work best bring that conversation back into view, through well-timed questions, plans the user can edit, and shared workspaces. The corpus says less about the reverse direction: how agents should summarize what they did so a user can actually check it afterward.
Sources 7 notes
Tool-enabled LLMs drift from user intent through silent tool chaining. Conversation analysis reveals insert-expansions—clarifying intent, scoping responses, enhancing appeal—as a formal framework for proactive user consultation that prevents misunderstanding instead of recovering from it.
Cocoa's system, which lets users assign plan steps to themselves or an agent and revise plans mid-execution, afforded greater steerability than chat-baseline interaction in a lab study (n=16) while maintaining ease of use, with field users valuing explicit work delegation.
JarvisHub proposes that placing prompts, references, versions, and feedback as typed canvas nodes visible to both users and agents—rather than hiding agent memory in chat or transient state—enables local updates, artifact reuse, and unfinished work continuation without process opacity.
SafeFlow demonstrates that multi-agent systems' core strength—splitting tasks and specializing roles—creates a safety blind spot where malicious intent can be distributed across steps that each appear benign individually, with harm emerging only in composition.
FLOWSTEER demonstrates that a crafted prompt can steer planner-executor systems by biasing workflow formation before infrastructure is invoked, raising malicious success by up to 55 percent. This attack surface exists because contamination enters upstream of workflow inspection defenses.
Show all 7 sources
Single task-success metrics obscure how agents achieve results across memory, context, and verification layers. Research shows identical success rates can mask enormous differences in efficiency, reliability, and deployment readiness—requiring harness-level benchmarks that measure trajectory, memory hygiene, and verification costs.
The AXIS framework shows that prioritizing API calls over sequential UI interactions cuts task completion time by 65–70% while maintaining 97–98% accuracy and reducing cognitive workload by 38–53%. A self-exploration mechanism automatically discovers and constructs APIs from existing applications, solving the bootstrapping problem.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Towards a Science of Scaling Agent Systems
- FLOWSTEER: Prompt-Only Workflow Steering Exposes Planning-Time Vulnerabilities in Multi-Agent LLM Systems
- SafeFlow: Semantic Information-Flow Control for Blocking Malicious Propagation in Multi-Agent Systems
- Cocoa: Co-Planning and Co-Execution with AI Agents
- From Model Scaling to System Scaling: Scaling the Harness in Agentic AI
- SoK: When Safe Agents Fail Together: The Security of Multi Agent LLM Systems
- Challenges in Human-Agent Communication
- Turn Every Application into an Agent: Towards Efficient Human-Agent-Computer Interaction with API-First LLM-Based Agents