Does letting users edit agent plans improve trust calibration?

An empirical study (N = 248) of "plan-then-execute" LLM agents handling everyday tasks like itinerary planning and currency transactions tested whether giving users the ability to edit an agent's step-wise plan, and to review or override its predicted actions during execution, would calibrate their trust in the system. It mostly did not. The authors report: "user involvement in the planning and execution does not significantly impact user trust and calibrated trust in planning and execution outcomes," and that their results "do not support H1 or H3, which indicates user involvement does not necessarily help calibrate user trust." Instead, a task-specific correlation analysis found that plan quality itself — not whether the user got to edit it — correlated positively with calibrated trust.

The mechanism the authors give is plausibility: LLM-generated plans "are formulated with a clear, logical structure, which covers most of the task requirements," so they "seem quite plausible and trustworthy" at first glance regardless of whether they are actually correct. Participant feedback illustrates the trap directly: "The plans look nice, I do not find any space for improvement" and "the planning stage of the LLM assistant was helpful and trustworthy" — comments made about plans the study's own quality scoring rated as flawed. The authors tie this to prior work on LLM-assisted fact-checking, where "convincing explanations provided by LLMs can cause over-reliance when LLMs are wrong."

This sharpens Does a single benchmark score actually predict agent readiness?: trust calibration is a separable axis from task completion, and this study shows the two can move independently — users can be given full edit access (nominally raising the "control" axis) while the "calibration" axis stays flat. It also qualifies Where do user values break down in agent supervision? and What makes people distrust AI agents they delegate to?: both of those studies locate trust problems at the moment of unauthorized or unreviewed action, implying that giving users a review step should help. This study complicates that assumption — a review step existed here (users could edit the plan before execution), and it still failed to fix trust calibration, because the failure mode is perceptual (plausible text misread as correct) rather than structural (no opportunity to intervene).

The study does not report what specifically made a plan "high-quality" versus "low-quality" in the excerpted text, nor the baseline accuracy of trust judgments without any plan access at all, so it cannot say whether giving users partial, summarized, or confidence-annotated views of a plan (rather than the full plausible text) would calibrate trust better than full edit access did. The implication the authors draw, and the excerpt supports, is narrower but still actionable: adding an edit affordance to an agent's plan is not itself a trust-calibration intervention, and designers who assume "let the user review it" closes the calibration gap should expect it not to, absent some signal that distinguishes a plausible plan from a correct one.