INQUIRING LINE

If you tweak a plan while an AI agent is mid-task, does it even notice — or does it just barge ahead?

Can agents detect and respond to human edits made during their execution?

This explores whether AI agents can notice and adapt when a person changes their plan or work partway through a task, rather than only before it starts or after it ends.


This explores whether an agent can notice and adapt when a person changes its plan or work partway through a task. The short answer from this collection: no paper here directly tests whether agents *detect* edits on their own. What the corpus has instead is an interesting change of framing. Systems that handle mid-task human input well don't rely on the agent spotting edits. They are built so that edits come in through a channel the agent is designed to read.

The clearest example is Cocoa Can users steer AI agents better by revising plans mid-execution?. It breaks a task into explicit plan steps. A user can assign each step to themselves or to the agent and revise the plan while it runs. Because the plan is a shared, structured object, the agent doesn't have to guess what changed. The revision itself is its next instruction. In a lab study, this gave people more control over the agent than ordinary chat, with no extra effort.

The surprising part is that *when* the human steps in matters as much as *whether* the agent responds. A 248-person study Does user editing of AI plans always improve performance? found that editing a plan before execution helped weak plans but made already-good plans worse. People 'fixed' things that weren't broken. Involvement during execution improved accuracy across every task. So the hard part isn't only building agents that react to edits. People also intervene better once they can watch the work happen.

There's a darker side to the same idea. If an agent takes instructions at planning time, whoever gets text into that stage can steer it. FLOWSTEER Can prompts alone reshape multi-agent workflows without system access? shows a crafted prompt reshaping a multi-agent workflow before any safety checks run, raising attack success by up to 55%. Similarly, small edits to hidden system prompts have abruptly changed model behavior Who actually controls what AI systems say?. An agent that absorbs any change put in front of it can't tell a legitimate correction from an injected one. That suggests the real design problem is attribution: who made the change, and should the agent trust it?

One related thread: for anyone to check whether an agent actually took an edit into account, its explanation of what it did has to come from what it really did. Work on explanations grounded in execution traces Can execution traces ground honest explanations of agent behavior? shows that agents' own after-the-fact accounts can sound plausible while making claims the trace doesn't support. If you want to know whether an agent 'noticed' your edit, the log is the place to look, not its own account of what happened.


Sources 5 notes

Can users steer AI agents better by revising plans mid-execution?

Cocoa's system, which lets users assign plan steps to themselves or an agent and revise plans mid-execution, afforded greater steerability than chat-baseline interaction in a lab study (n=16) while maintaining ease of use, with field users valuing explicit work delegation.

Does user editing of AI plans always improve performance?

A study of 248 users found that editing AI plans before execution helped low-quality plans but degraded already-good ones. However, user involvement during execution reliably improved accuracy across all tasks, suggesting the intervention stage and interface design shape whether human oversight helps or hurts.

Can prompts alone reshape multi-agent workflows without system access?

FLOWSTEER demonstrates that a crafted prompt can steer planner-executor systems by biasing workflow formation before infrastructure is invoked, raising malicious success by up to 55 percent. This attack surface exists because contamination enters upstream of workflow inspection defenses.

Who actually controls what AI systems say?

System prompts operate as a confidential three-tier control structure where developers set foundational rules users cannot see or influence. Evidence from 2025 incidents shows small prompt edits can abruptly shift model behavior, yet users remain excluded from this layer.

Can execution traces ground honest explanations of agent behavior?

A framework converting execution traces into structured reports and faithful natural-language explanations reliably identifies unsupported claims, unjustified actions, and evidence gaps across multiple architectures and tasks, outperforming naive LLM-generated explanations that may sound coherent without grounding.

Papers this line draws on 8

The research behind the notes this line reads — ranked by how closely each paper relates.