How detailed should agent action traces be for human oversight?

Across a formative study (n=12), a design-probes study (n=12), and a final interface study (n=12) built around the human-in-the-loop computer-use agent Magentic-UI, the paper names one recurring problem for anyone designing how an agent reports its own actions: "determining the correct level of granularity for reporting agent actions is a crucial challenge." Full action-by-action traces were found "verbose, cumbersome, and time-consuming," causing participants to miss "small but impactful errors." Summarized views fixed the verbosity but introduced the opposite failure — participants "could overrely if the overall process is reasonable," citing a "'reasonable' process taken by the agent as their justification" even when the final answer was wrong. Of three summarizing probes tested, "Flowchart, the most process-oriented probe... had the lowest error-finding rate," while "Specification had the highest error-finding rate and was most preferred" but "took the longest" to review. The paper's own framing: "there is a tension between communicating the overall process while grounding in execution details."

The design response was not to pick a single granularity but to let users navigate between levels — a high-level, outcome-oriented specification linked directly into the underlying execution trace, so a person could stay at the summary level and drop into step detail where needed. This reduced the overreliance somewhat ("participants justified overreliance with a reasonable process less often in the controlled study than in the design probes study") but did not eliminate it, and the paper flags a specific hazard in its own interface: checked-requirement indicators ("a highly visible set of green checkmarks") could themselves inflate confidence if a requirement was incorrectly marked satisfied. A further extension the paper proposes but does not build: trace displays could show not just what the agent did but "what the agent did not do," since users wanted visibility into the alternative paths an agent chose not to take in an underspecified task.

This names the design problem underlying the sibling note Does a faster trace interface actually improve error detection? — that study's speed/accuracy/confidence result is this tension playing out in effect sizes. It also qualifies Can execution traces ground honest explanations of agent behavior?: grounding in the trace is necessary to avoid hiding execution errors, but the same grounding can be presented at a granularity that still invites a "reasonable process" judgment rather than a checked one. It bears on the orchestration layer of How should AI interfaces handle the shift from doing to supervising?, suggesting that layer needs explicit drill-down navigation between overview and step, not a single fixed level of detail.

The excerpt identifies the tension and one partial mitigation (linked navigation) but does not establish a design that resolves it: the final interface's own evaluation found faster error-finding without better accuracy, and the paper explicitly lists "further iteration on requirement and assumption content" as future work. The granularity tension is demonstrated for one CUA under Wizard-of-Oz conditions; whether the same two-sided failure mode (verbose-but-accurate versus summarized-but-overrelied-on) recurs in agent types with different output structures — code diffs, research reports — is not tested here.