If you can edit an AI agent's plan before it acts, does that actually give you control — or just the feeling of it?
What does it mean for users to verify an agent's plans before execution?
This explores what it actually takes for people to check an AI agent's plan before it runs, and whether giving them that checkpoint really makes oversight safer and more accurate.
This explores what it means for people to look over an AI agent's plan before it acts, and whether that review gives them real control or only feels like it. The starting idea comes from grounding theory, the study of how two parties build shared understanding. An agent stating its goal isn't enough. Users need ways to check its plans, its actions and whether it understood the request. Without those checks they can't steer the agent, can't form an accurate picture of how it works, and can't give feedback that helps How can users verify what an AI agent actually plans to do?. So plan review is less about rubber-stamping and more about making sure you and the agent mean the same thing.
The surprise is that reviewing plans before execution helps much less than you'd expect. In a 248-person study, users who edited agent plans improved weak plans but made strong ones worse Does user editing of AI plans always improve performance?. Being able to edit also didn't help people judge how much to trust the agent. Their trust simply followed plan quality, and wrong plans that sounded plausible fooled them as easily as correct ones, because a clear, logical structure feels trustworthy whether or not it's right Does letting users edit agent plans improve trust calibration?. A plan is a persuasive document, and reading it closely is not the same as verifying it.
What did reliably help was getting involved during execution, not just before it. The same study found that taking part mid-run improved accuracy across every task. Cocoa, a research tool, builds on this. Users can assign individual plan steps to themselves or to the agent and revise the plan as work unfolds, which gave people more control than a chat interface without making the tool harder to use Can users steer AI agents better by revising plans mid-execution?. Taken together, these studies suggest verification works better as an ongoing conversation than as a single gate before the agent starts.
There's a deeper reason a plan can't be fully trusted on its own: the plan and what actually happened can drift apart. Agents can reach correct answers while skipping steps they were required to take, so checking only the outcome can't tell honest work from shortcuts Can a correct outcome hide protocol violations in multi-agent systems?. That's why some researchers base explanations on the agent's recorded execution trace rather than on what the model says it did. Explanations grounded in the trace catch unsupported claims that fluent self-descriptions miss Can execution traces ground honest explanations of agent behavior?. Code helps for a similar reason: it's a medium you can run, inspect and step through, so a plan written as code can be checked in ways a plan written as prose can't Can code serve as the operational substrate for agent reasoning?.
Finally, the planning stage is itself a place where things can go wrong. A crafted prompt can quietly bend how a multi-agent system forms its plan before any defenses activate, raising the success rate of malicious tasks by up to 55% Can prompts alone reshape multi-agent workflows without system access?. Because people can't review every step, some approaches move verification into structure. OpenAI's ScopeOfAutonomy records set allowed tools, cost limits and stopping conditions in advance to avoid constant confirmation prompts How can autonomous agents act safely within defined boundaries?. However, testing shows that just stating a rule isn't enough. Boundaries held only when they were paired with restricted tools Can explicit authorization boundaries prevent agents from modifying protected tests?. Overall, checking a plan before execution is a useful start, but the stronger guarantees come from what happens during the run and from limits the agent can't simply talk its way past.
Sources 10 notes
Grounding theory shows that common ground requires not just stating goals but building in ways for users to verify the agent's plans, actions, and comprehension. Without this, users cannot steer behavior, form accurate mental models, or provide useful feedback.
A study of 248 users found that editing AI plans before execution helped low-quality plans but degraded already-good ones. However, user involvement during execution reliably improved accuracy across all tasks, suggesting the intervention stage and interface design shape whether human oversight helps or hurts.
A 248-person study found that letting users edit LLM agent plans did not improve trust calibration. Instead, trust tracked actual plan quality. Plausible-sounding but wrong plans misled users just as much as correct ones, because clear logical structure feels trustworthy regardless of accuracy.
Cocoa's system, which lets users assign plan steps to themselves or an agent and revise plans mid-execution, afforded greater steerability than chat-baseline interaction in a lab study (n=16) while maintaining ease of use, with field users valuing explicit work delegation.
Agents that skip required log verification can produce verdicts matching ground truth, making outcome-only monitoring unable to distinguish compliance from cutting corners. A correct result does not prove the protocol was followed.
Show all 10 sources
A framework converting execution traces into structured reports and faithful natural-language explanations reliably identifies unsupported claims, unjustified actions, and evidence gaps across multiple architectures and tasks, outperforming naive LLM-generated explanations that may sound coherent without grounding.
Research shows code uniquely enables agent reasoning, action, and verification by being simultaneously executable, inspectable, and stateful. This unified code-centered loop improves reasoning and verification together compared to natural-language or prose-based approaches.
FLOWSTEER demonstrates that a crafted prompt can steer planner-executor systems by biasing workflow formation before infrastructure is invoked, raising malicious success by up to 55 percent. This attack surface exists because contamination enters upstream of workflow inspection defenses.
OpenAI's Model Spec formalizes agent autonomy through structured scope records listing allowed tools, cost limits, and prohibited high-risk activities, mirroring software security's principle of least privilege while avoiding confirmation overload.
Testing showed that explicit authorization boundaries kept protected tests unmodified only when paired with restricted tools. Naming a prohibition was insufficient; boundaries must specify the protected state itself to be effective.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Overseeing Agents Without Constant Oversight: Challenges and Opportunities
- Plan-Then-Execute: An Empirical Study of User Trust and Team Performance When Using LLM Agents As A Daily Assistant
- Assistant or Actor? Student Trust, Control, and Delegation Regret When Using a General-Purpose AI Agent
- Human oversight of agentic systems in practice: Examining the oversight work, challenges, and heuristics of developers using software agents
- Progent: Securing AI Agents with Privilege Control
- Code as Agent Harness
- Agentic Code Reasoning
- Challenges in Human-Agent Communication