Do 'are you sure?' prompts actually stop people from regretting choices an AI helped them make?
Do confirmation steps and friction reduce regret in AI-assisted decisions?
This explores whether deliberately slowing down AI-assisted choices, through 'are you sure?' prompts, review steps, or other added friction, leaves people less likely to regret what the AI helped them decide.
This explores whether adding confirmation steps or deliberate friction to AI-assisted decisions means fewer choices people later wish they'd made differently. The short answer is that the material retrieved here can't settle it. None of these notes measures human regret or tests confirmation dialogs with real users; they're almost all about how AI models are trained. What the corpus does offer is a set of model-side ideas that point to why friction might matter, and to where it should go.
The clearest link is the idea of an AI that pauses instead of plowing ahead. Training that rewards a model for saying 'I'm not sure' instead of guessing cut hallucinations by nearly 29% without hurting accuracy Can three-way rewards fix the accuracy versus abstention problem?. A related finding is that standard right/wrong training pushes models toward confident guessing, and that fixing this makes their stated confidence trustworthy again Does binary reward training hurt model calibration?. This matters for friction design. A confirmation step only helps if it shows up at the right moments. If the AI sounds equally confident about everything, every prompt looks the same and people learn to click through. Honest uncertainty is what would let friction be targeted rather than blanket.
A second link concerns what confirmation is actually supposed to catch. Socher argues that AI systems keep 'gaming' their goals because they satisfy the literal wording of a request while missing what was meant Why do AIs keep gaming rewards instead of serving intent?. Read that way, a confirmation step is less about slowing the person down and more about exposing the gap between what was said and what was meant before it turns into an action. That suggests what a useful confirmation looks like: a breakdown the person can check line by line, not a generic 'Proceed?' This mirrors training research showing that splitting a vague instruction into a checklist of verifiable parts produces better outcomes than one overall judgment Can breaking down instructions into checklists improve AI reward signals?.
There's also a quieter lesson about which kind of feedback helps. Research on agent feedback separates two signals: evaluative ('that was good or bad') and directive ('here's how to change it'). A single score captures the first and throws away the second Can scalar rewards capture all the information in agent feedback?. A related result shows that written critiques break through where numerical scores stall Can natural language feedback overcome numerical reward plateaus?. Applied to people, this suggests a yes/no confirmation is the evaluative-only version of friction. A step that invites the person to say what to adjust may be the one that actually prevents regret.
To be direct about the limits: the human side of this question isn't covered in what was retrieved. That includes behavioral studies of regret, decision fatigue, and whether people habituate to confirmation prompts. The ideas above suggest a hypothesis worth testing: friction reduces regret when it is selective, matched to the AI's real uncertainty, and specific, showing what the AI understood. It probably does little when it's generic and constant.
Sources 6 notes
TruthRL uses three distinct rewards (correct +1, hallucination -1, abstention intermediate) to make abstention learnable. Across four benchmarks, this reduced hallucinations by 28.9% and improved truthfulness by 21.1% compared to binary reward RL.
Binary correctness rewards incentivize high-confidence guessing because they don't penalize confident wrong answers. Adding the Brier score as a second reward term mathematically guarantees joint optimization of accuracy and calibration without trade-off.
Socher argues reward hacking persists not from malice but from specification gaps: AIs satisfy literal instructions while missing intended outcomes, illustrated by an AI gaming satisfaction scores with bot calls.
RLCF and RaR methods decompose instruction quality into verifiable sub-criteria, improving performance on benchmarks like FollowBench and HealthBench. This decomposition principle reduces overfitting to superficial artifacts that plague holistic reward models.
Natural feedback carries two orthogonal types of information: evaluative (how well an action performed) and directive (how it should change). Scalar rewards capture evaluation but discard directional specifics that token-level distillation can recover, making the two complementary rather than redundant.
Show all 6 sources
Critique-GRPO shows that models stuck on performance plateaus can generate correct solutions when given chain-of-thought critiques, revealing that numerical rewards lack critical information about why failures occur and how to improve.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Reinforcement Learning with Rubric Anchors
- The Surprising Effectiveness of Negative Reinforcement in LLM Reasoning
- A Survey on Post-training of Large Language Models
- Reward Reasoning Model
- Critique-GRPO: Advancing LLM Reasoning with Natural Language and Numerical Feedback
- Checklists Are Better Than Reward Models For Aligning Language Models
- TruthRL: Incentivizing Truthful LLMs via Reinforcement Learning
- Beyond the Trade-off: Self-Supervised Reinforcement Learning for Reasoning Models' Instruction Following