Could an AI agent flag how confident it is in a plan before it hands you the pen to edit it?
Can system interfaces signal plan confidence before inviting user edits?
This explores whether an AI agent could tell users how confident it is in a plan before asking them to review or edit it, so people put their effort into the plans that need it and leave the good ones alone.
This explores whether an interface could show how sure an agent is about its plan before it invites the user to make changes. The corpus has no study that tests this exact design. It does have enough nearby evidence to show why the idea matters and where it could go wrong. Start with the problem it would solve. A 248-person study found that editing an agent's plan before it ran improved weak plans but made already-good plans worse Does user editing of AI plans always improve performance?. The same study found that having edit access didn't help people judge when to trust the agent. Their trust followed the plan's actual quality, and wrong plans that sounded plausible fooled them just as often as correct ones, because a clear, logical structure feels trustworthy whether or not it's right Does letting users edit agent plans improve trust calibration?. So people are poor at telling a good plan from a bad one by reading it. A confidence signal could fill that gap and point their edits where they help.
The catch is that confidence signals are persuasive in their own right. Research across many languages found that users everywhere follow how confident the AI sounds rather than whether it's accurate, so overconfident mistakes get followed reliably Do users worldwide trust confident AI outputs even when wrong?. A confidence label on a plan could make the problem worse. If the signal is miscalibrated, a high-confidence badge on a bad plan would discourage the very edits that would have fixed it. A confidence signal is only safe when it is accurate.
The corpus offers some leads on how to make it accurate. One approach grounds confidence in the model's track record: it looks up past episodes where the model had similar confidence and checks how often it was actually right. This matched expensive repeated-sampling methods at about a tenth of the cost, and the signal came entirely from those stored outcomes Can past performance predict when a model will be right?. A cheaper check is stability. When a model is confident, its output holds steady when the prompt is reworded, and low confidence shows up as big swings Does model confidence predict robustness to prompt changes?. A plan that changes a lot when the request is rephrased is a natural one to flag for review.
The plan-editing study also points to a different framing. Users who stepped in while the plan was running improved accuracy on every task Does user editing of AI plans always improve performance?. That suggests showing confidence step by step as the plan runs, rather than as one score up front. Conversation analysis gives agents a formal way to decide when to stop and ask the user a clarifying question instead of quietly chaining tools and drifting from what the user meant When should AI agents ask users instead of just searching?. In that view, low confidence is a reason to ask the user a specific question at that step, not just a number on the screen.
Here's what you may not have expected. If confidence signals worked, they wouldn't mainly make users trust the agent more. They would get users to edit less, holding back on good plans and stepping in only where the agent is unsure. Whether interfaces can do this without simply moving people's overreliance from how confident the writing sounds to a confidence badge is still an open question in this collection.
Sources 6 notes
A study of 248 users found that editing AI plans before execution helped low-quality plans but degraded already-good ones. However, user involvement during execution reliably improved accuracy across all tasks, suggesting the intervention stage and interface design shape whether human oversight helps or hurts.
A 248-person study found that letting users edit LLM agent plans did not improve trust calibration. Instead, trust tracked actual plan quality. Plausible-sounding but wrong plans misled users just as much as correct ones, because clear logical structure feels trustworthy regardless of accuracy.
Cross-linguistic research shows users in every language trust confident AI outputs even when inaccurate. While confidence expression varies by language, users everywhere track confidence signals rather than accuracy, making overconfident errors systematically followed.
XConf matches ten-sample self-consistency at a tenth of the cost by retrieving the model's past episodes with similar confidence levels and reading their historical success rates. Ablations show the signal depends entirely on stored outcomes, not on the retrieval prompt itself.
ProSA found that when models are highly confident, they resist prompt rephrasing; low confidence causes major output swings. Larger models, few-shot examples, and objective tasks all correlate with higher confidence and greater robustness.
Show all 6 sources
Tool-enabled LLMs drift from user intent through silent tool chaining. Conversation analysis reveals insert-expansions—clarifying intent, scoping responses, enhancing appeal—as a formal framework for proactive user consultation that prevents misunderstanding instead of recovering from it.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Reported Confidence in LLMs Tracks Commitment More Than Correctness
- What Large Language Models Know and What People Think They Know
- Post-Training Large Language Models via Reinforcement Learning from Self-Feedback
- Plan-Then-Execute: An Empirical Study of User Trust and Team Performance When Using LLM Agents As A Daily Assistant
- Confidence Comes from Experience: Experiential Confidence Estimation from Reasoning to Agents
- Humans overrely on overconfident language models, across languages
- Insert-expansions For Tool-enabled Conversational Agents
- Intent Mismatch Causes LLMs to Get Lost in Multi-Turn Conversation