TOPIC
Decision Support Tools
A subject the collection covers, read through 6 synthesis notes.
View as
Do cognitive forcing functions help all decision makers equally?
Research explores whether interventions designed to reduce overreliance on AI assist everyone fairly, or whether benefits concentrate among those already motivated to think hard.
Do forcing functions reduce AI overreliance without harming team performance? Does using LLMs actually improve strategic decision making? Can AI guidance reduce anchoring bias better than AI decisions? When do human-AI teams actually outperform either alone? How do cognitive tools reshape the three types of worker skill?
Cognitive forcing functions reduce overreliance but worsen team performance and user acceptance LLM use reshapes strategic thinking without improving forecast accuracy AI guidance replaces deference by highlighting useful input aspects for human decisions Human-AI teams underperform the best alone except in creation tasks Cognitive tools substitute for implementation skill but complement judgment
Which individual traits predict who benefits from AI decision support tools? Can metacognitive feedback reduce how much people offload thinking to AI? Can forcing functions be redesigned to help low-motivation participants too? Do younger people show different cognitive offloading patterns with AI tools? Does intervention friction in decision support reduce cognitive immersion for users? Can interventions addressing AI hallucination and bias prevent cognitive offloading? Why do decision-support systems reduce overreliance without improving team performance?
Do forcing functions reduce AI overreliance without harming team performance?
This study tests whether cognitive forcing functions—interventions that disrupt quick reliance on AI suggestions—can reduce overreliance while maintaining or improving human-AI team performance on decision tasks.
Does AI assistance always help reasoning or does it carry hidden costs? Can AI guidance reduce anchoring bias better than AI decisions? Why don't radiologists benefit from AI predictions? Do cognitive forcing functions help all decision makers equally? When do human-AI teams actually outperform either alone?
AI reasoning interventions degrade cognitive flow even when they are correct AI guidance replaces deference by highlighting useful input aspects for human decisions Radiologists underweight AI predictions due to belief-updating errors Cognitive forcing functions disproportionately benefit high-motivation thinkers Human-AI teams underperform the best alone except in creation tasks
Can forcing functions be redesigned to help low-motivation participants too? Does intervention friction in decision support reduce cognitive immersion for users? Can interventions addressing AI hallucination and bias prevent cognitive offloading? Why do decision-support systems reduce overreliance without improving team performance? How does AI intensify work demands even when it doesn't reduce headcount? What negative cycles emerge when organizations overemphasize one approach alone? How does automating information acquisition differently affect mental workload and situation awareness? Can hiding AI advice from users reduce overreliance without hurting team performance? What role does information asymmetry play in human-AI team performance?
Why do readers trust LLM explanations more than they should?
Exploring why human confidence in LLM answers diverges so sharply from the model's own internal confidence signals. This matters because users may make poor decisions based on explanations that feel credible but misrepresent actual accuracy.
Do users worldwide trust confident AI outputs even when wrong? Does verbal confidence actually predict answer correctness? Why do language models fail confidently in specialized domains? Does longer explanation text actually improve answer evaluation? Do LLM explanations actually help people judge answer quality?
Users universally overrely on overconfident LLM outputs across all languages Verbal confidence predicts model decisions, not answer correctness Language models show persistent overconfidence in specialized knowledge domains Longer explanations boost confidence without improving answer discrimination Explanations boost trust in both correct and incorrect LLM answers
Why do people trust explanations as signals of competence rather than reading them closely? Does verbal confidence in LLMs track decision-making better than actual correctness? Can longer explanations make readers more confident without improving answer accuracy? How persistent is LLM overconfidence in specialized knowledge domains? Why do surface features like text length drive trust more than accuracy signals? How does verbal uncertainty language affect human reliance on LLM outputs? What calibration gap exists between human confidence and actual answer correctness? Why do LLM explanations feel convincing even when the underlying plan is wrong? Why do summaries of agent behavior invite false confidence in wrong answers? Can source citations help people catch when LLMs provide incorrect information? How do explanation length and confidence language affect human trust in LLM outputs? Why do people rely on surface features like fluency over actual accuracy?See all 15 inquiring lines on this note →
When do human-AI teams actually outperform either alone?
Most human-AI combinations underperform the best of human or AI solo, but the pattern differs by task type. Understanding when and why collaboration works matters for designing effective AI partnerships.
Do forcing functions reduce AI overreliance without harming team performance? Why don't radiologists benefit from AI predictions? Can generative AI replace the benefits of having a human teammate? Does theory of mind predict who thrives in AI collaboration? Does AI help or harm consultant performance on unfamiliar tasks?
Cognitive forcing functions reduce overreliance but worsen team performance and user acceptance Radiologists underweight AI predictions due to belief-updating errors Generative AI matched individual performance to teams without AI at Procter & Gamble Theory of mind predicts AI collaboration ability independent of solo performance AI assistance reversed consultant performance outside its capability frontier
Can a machine be cooperative in one routine but merely assisting in another? When can aggregating human and AI decisions outperform either alone? Why do human-AI combinations underperform on decision tasks but succeed in creation? Does human-AI ensembling add value even when neither performer is individually accurate? Where does AI fit on the spectrum from passive tool to cooperative partner? Why do most human-AI decision teams underperform the better individual alone?
Does longer explanation text actually improve answer evaluation?
When LLMs provide longer explanations, do people get better at spotting wrong answers, or does the extra length just make them feel more confident? This matters because interface design often assumes more explanation equals better understanding.
Do users worldwide trust confident AI outputs even when wrong? Does verbal confidence actually predict answer correctness? Why do readers trust LLM explanations more than they should? Do LLM explanations actually help people judge answer quality? Do sources alone help readers catch LLM errors better than sources plus explanations?
Users universally overrely on overconfident LLM outputs across all languages Verbal confidence predicts model decisions, not answer correctness Default LLM explanations create large calibration gaps between human and model confidence Explanations boost trust in both correct and incorrect LLM answers Sources without explanations outperform sources with explanations for catching incorrect LLM answers
Do reflection questions help people make better decisions with AI?
This explores whether conversational AI that prompts users to think through problems outperforms AI that simply provides answers. Understanding this matters for designing AI tools that genuinely improve human judgment rather than replace it.
Models can be trained to proactively identify missing information and request clarification AI passivity stems from next-turn reward optimization, not inability Conversational AI agents are structurally passive and reactive by design AI passivity in conversation stems from training incentives, not capability limits Insert-expansions from conversation analysis formalize when agents should probe users
What are the five specific conversation triggers where AI intervention adds value? How does AI assistance differ from search engines in cognitive impact? Does the timing of AI feedback relative to user reasoning change its effectiveness? Can users tell the difference between their own thinking and AI contribution? How do contrasting examples improve AI feedback quality over generic suggestions? What distinguishes reflection that satisfies constraints from reflection that merely sounds reflective? Why do conversational systems benefit from post-thinking between user turns? How does AI assistance affect human cognitive development over time? How does AI assistance change learning outcomes across different cognitive engagement levels? Can explicit reflection during AI-assisted work improve transfer of learning? Why do users prefer AI responses that actually harm their decision-making? How might automated evals eventually capture the human judgment designers exercise now?See all 39 inquiring lines on this note →