When should agents ask before acting on ambiguous commands?

When a user's command is ambiguous about which UI element it refers to, the paper's Wizard-of-Oz findings show that tolerance for autonomous resolution depends on stakes. "Users may prefer the agent to make reasonable assumptions automatically when the risk of taking a diverting UI path is low, such as navigating to a webpage section through search instead of through navigation menu." But "in high-stakes or impactful situations, such as those involving payments, users do not want the agent to act automatically and instead prefer to be informed of possible options and have the agent request clarification before proceeding with an action." Ambiguity itself, the paper notes, "may stem from different levels of user expression (e.g., expressing intent rather than gestures) or from the user's goals (e.g., during exploratory UI tasks)."

The paper frames this as an extension of Horvitz's mixed-initiative principles for interface agents, which held that agents should handle ambiguity by "considering uncertainty about a user's goals," "inferring ideal action in light of costs, benefits, and uncertainties," and "employing dialog to resolve key uncertainties." The authors write that their Wizard-of-Oz findings "echo this early vision, and we extend it by providing details in the context of (M)LLM-based computer use agents" — naming payments specifically as the kind of action where cost and consequence outweigh the convenience of autonomous action.

This gives the library's control and oversight literature a stakes-contingent mechanism rather than a flat preference for autonomy or control. How can users verify what an AI agent actually plans to do? establishes that verification matters generally; this finding specifies when it matters most — rising with stakes rather than applying uniformly. What UX principles do workplace users want in AI agents? reports human control as a standing priority among business users, while this finding suggests that priority is conditional, rising and falling with the impact of the action in question. Where do user values break down in agent supervision? likewise ties the need for supervision to the conditions around a run; here the condition is named explicitly as stakes, exemplified by payment actions. The finding is one instance of the broader What design areas matter most for computer use agent UX? this study produced.

The finding comes from a researcher-simulated agent, not a working model, and payments is the excerpt's only named example of a "high-stakes" action — the paper does not say how broadly the stakes threshold generalizes to other consequential actions (deleting data, sending messages, changing settings), nor does it quantify how many of the 20 participants expressed this preference. It also does not resolve how an agent should estimate risk or stakes in the moment, only that users want that estimate reflected in how much autonomy the agent takes. At the strength the excerpt supports, a single fixed autonomy setting for ambiguity resolution is likely to feel wrong at one end of the stakes spectrum or the other; the open design question is how an agent infers stakes without asking the user to specify it every time.