TOPIC
Human-Centered Design
A subject the collection covers, read through 43 synthesis notes.
View as
Is AI shifting from message conduit to active conversation participant?
HCI researchers may be reconceiving AI's role in human-to-human communication, moving beyond passive formatting toward active participation. This matters as systems grow more capable post-2023.
Machine agency operates on a five-level spectrum from passive to cooperative Conversational AI agents are structurally passive and reactive by design Proactive agents need civility design alongside intelligence and adaptivity AI agents are passive by design, not by capability Transactional AI agents co-construct consumer preferences through repeated conversation loops
How should AI systems be designed across different interaction phases?
Can a small set of general guidelines help practitioners design better AI-infused products across different moments—initial setup, active use, error handling, and long-term adaptation?
How should AI interfaces handle the shift from doing to supervising? What UX principles do workplace users want in AI agents? Does granting agents more autonomy undermine human oversight? Why don't design recommendations transfer cleanly to AI companions? What design areas matter most for computer use agent UX?
Mature AI interfaces require three distinct layers handling different failure modes Workplace users prioritize control, reliability, context-awareness, and safety in AI agent design Agent autonomy degrades human oversight capacity and positioning Ethics and UX considerations are inseparable in AI companion design Computer use agent UX organizes into four design areas with 21 subcategories
Who bears responsibility when AI seems human-like?
Does human-likeness in AI come from how users perceive systems or how designers build them? Understanding this distinction clarifies where accountability lies when AI causes harm.
Three cognitive traps compound into epistemic drift in human-AI interaction AI disclosure triggers short-term bias that reverses with repeated outcome feedback AI companionship emerges through functional use and adopts human relationship customs LLMorphism spreads through analogical transfer and metaphorical saturation Absence of human judgment enables both deeper disclosure and dishonesty
What responsibility do designers bear for consciousness attribution risk? What measurable harms occur when users interact with AI as if it were conscious? Can design choices reduce harm without resolving the consciousness question? How does the philosophical distinction between simulation and realization affect liability? What are the three dimensions of anthropomimesis and their harms? When both anthropomorphism and anthropomimesis occur together, which should we address first? How do humans and AI develop accurate models of each other? What happens to human expectations when they mistake consistent AI behavior for human behavior? What design signals help users know when AI is acting on their behalf? How should systems design transparency to make human-machine contribution boundaries visible? How do courts assign liability when AI intermediaries cause harm to consumers? When should a company be responsible for an AI system's errors?See all 20 inquiring lines on this note →
Does AI autonomy change how we assign blame?
When humans and AI jointly cause harm, do people blame them differently based on how much autonomy the AI had? Understanding this matters for designing fair accountability systems.
Does decomposing AI systems shift where people assign blame? Do automated systems shift blame onto their human operators? Do humans mistake AI kindness for human generosity in mixed groups? Do overseers focus on agent execution or action safety? Does attributing agency to AI actually reduce human sense of control?
Decomposing AI shifts causal blame toward the agentic component Automated systems shift failure blame onto operators with limited control Humans misattribute AI prosociality to humans, corrupting models of human behavior Overseers judge agent procedure over action safety Attributing agency to AI displaces human control through reversible recursive cycles
What happens to accountability when AI and humans share agency? Do people blame nearby humans more when systems fail automatically? Is autonomy legally required for holding AI systems responsible? How does physical proximity to harm affect causal blame assignment? Do people judge AI more causally responsible when it has greater autonomy? How does perceived autonomy affect third-party judgments of AI versus human responsibility? Why do people credit themselves for AI successes but blame AI for failures?
Should each automated function have its own independent level?
Can a single system balance different automation levels across information acquisition, analysis, decision-making, and action execution? Understanding function-specific automation may reveal how to optimize human performance across different task stages.
Machine agency operates on a five-level spectrum from passive to cooperative AI autonomy is a design choice separable from agent capability Agent autonomy degrades human oversight capacity and positioning Automation harm concentrates when control shifts to action selection Automation and human control are independent design axes not a trade-off
How do anchoring and Dunning-Kruger deepen AI overreliance?
Does automation bias persist not just from lack of information, but through active cognitive mechanisms like anchoring effects and self-overestimation? Understanding these compounding biases could reveal why transparency alone fails to reduce overreliance.
Does explaining AI decisions actually reduce user overreliance on it? Do users worldwide trust confident AI outputs even when wrong? Does unmodified chatbot behavior block rule discovery through sampling bias? How much does wrong AI advice harm radiologist accuracy? Do forcing functions reduce AI overreliance without harming team performance?
Explainable AI often deepens automation bias instead of reducing it Users universally overrely on overconfident LLM outputs across all languages Unmodified chatbots suppress rule discovery as much as explicit sycophantic prompting Wrong AI suggestions sharply reduce radiologist diagnostic accuracy. Cognitive forcing functions reduce overreliance but worsen team performance and user acceptance
Why do explanations and transparency often fail to reduce automation bias? How do confidence signals in AI outputs drive anchoring effects? Does confirmatory sampling toward initial hypotheses mirror automation bias? How do AI literacy and professional expertise shape vulnerability to automation bias? Can trace exposure fix overconfidence in human self-evaluation?
Who watches the automatic alarm when it fails?
Bainbridge identified a structural problem in delegating vigilance to automation: if humans cannot sustain attention to rare events, who monitors the automated system meant to replace that watching?
Why do safety failures remain invisible to our evaluation methods? Can organizations lose scrutiny capacity while keeping oversight forms? Does automation actually remove humans from the control loop? Does increased automation erode human oversight ability? What human limits actually undermine AI oversight today?
Safety failures hide in evaluation habits, not in technical obscurity Organizations can maintain nominal oversight while losing actual scrutiny capacity Automation leaves operators with harder tasks and eroded skills Automation creates three recurring costs for human operators. Human oversight of AI fails due to cognitive limits and misaligned incentives
Does automation actually remove humans from the control loop?
Bainbridge's 1983 research asks whether automating away the operator really eliminates human involvement, or whether it leaves them with harder tasks while their skills decay from disuse.
Does granting agents more autonomy undermine human oversight? Can organizations lose scrutiny capacity while keeping oversight forms? Does AI augmentation protect workers from skill erosion? Does increased automation erode human oversight ability? When does automation harm performance after failures?
Agent autonomy degrades human oversight capacity and positioning Organizations can maintain nominal oversight while losing actual scrutiny capacity Augmentation carries hidden risk through gradual skill and oversight erosion Automation creates three recurring costs for human operators. Automation harm concentrates when control shifts to action selection
How does automation erode operator skills needed to catch its own failures? How do manual control skills degrade when operators stop using them regularly? What happens to human oversight skills when automation increases over time? How does extended use of automated systems degrade operator skills? Why do ATMs and automated systems often complement rather than replace human workers?
Does increased automation erode human oversight ability?
Bainbridge's 1983 analysis identified three costs of automation in aviation and process control: vigilance decrement, takeover difficulty, and de-skilling. Does this pattern hold for AI systems and human overseers today?
Does granting agents more autonomy undermine human oversight? Does AI augmentation protect workers from skill erosion? Does targeted human oversight beat both full autonomy and exhaustive review? Should AI systems be designed as subservient co-pilots? Does automation actually remove humans from the control loop?
Agent autonomy degrades human oversight capacity and positioning Augmentation carries hidden risk through gradual skill and oversight erosion Targeted intervention at high-uncertainty points outperforms both full autonomy and exhaustive oversight The co-pilot metaphor positions AI as subservient to human direction and control. Automation leaves operators with harder tasks and eroded skills
How does automation erode operator skills needed to catch its own failures? How do manual control skills degrade when operators stop using them regularly? Do AI users show the same vigilance decrement as pilots? Can advanced automation systems require better human oversight than simpler ones? How does situational awareness affect pilot response to automation failure? Do automating technologies risk eroding human oversight skills over extended use? What happens to human oversight skills when automation increases over time? What causes automation to make easy tasks easier but hard tasks harder? Why does monitoring autonomous systems require more human effort and attention? How does extended use of automated systems degrade operator skills?
Can human-AI research teams improve faster than autonomous AI systems?
Explores whether keeping humans actively involved in AI research collaboration accelerates paradigm discovery compared to fully autonomous self-improvement, and what safety advantages this preserves.
What limits how much models can improve themselves? Does incremental AI replacement erode human influence over society? What capabilities do AI systems need for autonomous science? Can AI systems improve their own learning strategies? Do autonomous research mechanisms work better together than apart?
Self-improvement is fundamentally bounded by the generation-verification gap Incremental AI adoption degrades implicit alignment by replacing human labor dependency. Autonomous science requires four capabilities beyond current AI benchmarks Self-improving agents need intrinsic metacognition to adapt learning strategies Autonomous research mechanisms are complementary with super-additive interaction effects
What separates performative behavioral change from actual capability development in AI? Which workplace tasks see productivity gains when AI and users align? How does the ideation-execution gap differ between AI and human-generated research? Does removing human labor from systems secretly grant AI more autonomy? How do goal representations differ between human and AI teams? Where do human researchers retain competitive advantage over autoresearch systems? Why do major AI breakthroughs require human-discovered data and method combinations? Can humans build reliable oversight for increasingly complex AI systems? What implicit alignment do humans provide by staying in research loops? Which research collaboration skills should AI systems develop first? Why did every major AI paradigm require human data and method innovation? What role does evaluation play in human-AI creative collaboration?See all 118 inquiring lines on this note →
What design areas matter most for computer use agent UX?
As computer use agents become more common, what aspects of their design and interaction do users actually care about? A formative study explored this gap by analyzing real agents and testing design dimensions with users.
How should AI systems be designed across different interaction phases? What UX principles do workplace users want in AI agents? How should AI interfaces handle the shift from doing to supervising? How can users verify what an AI agent actually plans to do? When should agents ask before acting on ambiguous commands?
Eighteen guidelines for human-AI interaction organize by phase in the user journey. Workplace users prioritize control, reliability, context-awareness, and safety in AI agent design Mature AI interfaces require three distinct layers handling different failure modes Users need verification mechanisms to establish shared understanding with AI agents Users want agents to auto-resolve low-stakes ambiguity but confirm before high-stakes actions
Does decomposing AI systems shift where people assign blame?
When an AI system is split into a language model and an agentic tool-using layer, do people attribute more causal responsibility to one component over the other? This matters for understanding how liability and oversight might be distributed in real systems.
AI blame rises with autonomy, yet humans stay favored Role-play collapses into real agency once tools produce consequences Human-in-the-loop design treats intervention as oversight while neglecting knowledge and observation Attributing agency to AI displaces human control through reversible recursive cycles AI agents are passive by design, not by capability
What happens to accountability when AI and humans share agency? How do the four archetypes of agentic systems differ in where they place agency? Do people blame nearby humans more when systems fail automatically? Is autonomy legally required for holding AI systems responsible? How does physical proximity to harm affect causal blame assignment? Do people judge AI more causally responsible when it has greater autonomy?
What misconceptions hide in how we describe large language models?
Both deflationary framings like "just autocomplete" and anthropomorphic claims about emerging agency capture something real about LLMs, but each may overextend its truth. What distinctions help separate genuine features from overreach?
Three cognitive traps compound into epistemic drift in human-AI interaction LLM text generation and human communication are structurally different operations LLM outputs are subjective priors, not empirical observations. LLMs share cognitive structure with humans but lack socialization properties Large language models likely embody new non-causal modes of understanding
What makes an AI a true thought partner, not just a tool?
Can AI systems be designed to understand users, act transparently, and share mental models with humans? This explores whether current scaling approaches miss cognitive requirements for genuine partnership.
Does theory of mind predict who thrives in AI collaboration? What breaks when humans and AI models misunderstand each other? Can AI decompose social reasoning into distinct cognitive stages? Can AI agents communicate efficiently in joint decision problems? Why do LLMs excel at social norms yet fail at theory of mind?
Theory of mind predicts AI collaboration ability independent of solo performance Mutual theory of mind requires bidirectional updating to prevent material misalignment Decomposing social reasoning into stages achieves human-level theory of mind Decision-oriented dialogue requires agents to actively structure information exchange LLMs excel at social norm prediction but fail at genuine mental state reasoning
Why can't users and AI articulate shared goals together? What interpretive work must humans perform to experience AI as a conversation partner? Can people form genuine bonds with partners they know are not human? What would genuine semiosis require in an artificial system? How do humans and AI develop accurate models of each other? What individual differences predict who benefits from AI partnership? Which research collaboration skills should AI systems develop first? Can reward engineering and information-theoretic architecture solve partner-awareness separately? Why do some occupations need human-AI partnership more than others? Can robots with sensors create the shared world that consciousness requires? How does theory of mind predict success in human-AI partnerships? How does theory of mind predict who benefits from AI collaboration?See all 43 inquiring lines on this note →
Does explaining AI decisions actually reduce user overreliance on it?
Explainable AI is designed to build appropriate trust, but does transparency about how AI reaches recommendations actually help users catch mistakes and avoid automation bias, or does it backfire?
Do forcing functions reduce AI overreliance without harming team performance? Does access to web search prevent overreliance on chatbots? Do users worldwide trust confident AI outputs even when wrong? How do anchoring and Dunning-Kruger deepen AI overreliance? Do LLM explanations actually help people judge answer quality?
Cognitive forcing functions reduce overreliance but worsen team performance and user acceptance Prior trust in chatbots drives verification more than answer correctness. Users universally overrely on overconfident LLM outputs across all languages Anchoring and Dunning-Kruger compound automation bias to deepen overreliance Explanations boost trust in both correct and incorrect LLM answers
Why do explanations and transparency often fail to reduce automation bias? Can hiding AI advice from users reduce overreliance without hurting team performance? Can AI explainability help humans and machines reach better joint decisions? Can explanation alone prevent workers from over-relying on AI systems?
Where does the meaning of an AI explanation actually come from?
Does a single user reading an explanation create its meaning, or does meaning emerge from the social layers surrounding that reading—colleagues' interpretations, organizational norms, public discourse?
Language models learn culturally situated discourse patterns, not abstract language XAI effectiveness emerges from the source-framing-recipient triad AI claims proliferate without the conversational mechanisms that make knowledge reliable AI conversation lacks the ritual machinery that sustains human social order. LLMs share cognitive structure with humans but lack socialization properties
Why do stakeholders interpret the same explanation differently in practice? How does the evaluator become part of the definition of intelligence? How do organizational roles and peer interpretations shape what an explanation means? What specific signals would be needed for an AI system to acquire meaning? What role does Peirce's semiotic framework play in understanding AI meaning? Why does explanation source matter more than explanation content? Does removing information about who wrote something change how we interpret it? How does AI knowledge become structurally different from written sources? What explanation format actually helps users detect errors in AI systems? How should AI explanations be evaluated as human interfaces rather than model properties? Do AI rationales explain why a choice was made or justify it afterward? What makes a scientist satisfied with an explanation versus just a prediction?See all 15 inquiring lines on this note →
Do LLM explanations actually help people judge answer quality?
When large language models provide explanations, do people become better or worse at recognizing wrong answers? A controlled experiment tests whether explanations improve trust appropriately or enable overreliance.
Does longer explanation text actually improve answer evaluation? Do users worldwide trust confident AI outputs even when wrong? Do sources alone help readers catch LLM errors better than sources plus explanations? Why do readers trust LLM explanations more than they should? Does explaining AI decisions actually reduce user overreliance on it?
Longer explanations boost confidence without improving answer discrimination Users universally overrely on overconfident LLM outputs across all languages Sources without explanations outperform sources with explanations for catching incorrect LLM answers Default LLM explanations create large calibration gaps between human and model confidence Explainable AI often deepens automation bias instead of reducing it
Can source citations help people catch when LLMs provide incorrect information? How do explanation length and confidence language affect human trust in LLM outputs? What contexts beyond question answering might show different LLM explanation effects? Can trace exposure fix overconfidence in human self-evaluation? How much does LLM explanation length affect reader confidence in wrong answers? Do explanations and sources have additive or offsetting effects on user trust?
Can models express uncertainty instead of just answering?
Most factuality work expands what models know rather than what they know they know. Can expressing calibrated uncertainty create a third path between confident errors and unhelpful abstention?
Does reasoning fine-tuning make models worse at declining to answer? Can a model be truthful without actually being honest? Can LLM explanations actually help humans predict model behavior? Can models learn to judge their own performance accurately? Do LLMs overgeneralize when summarizing scientific research?
Reasoning fine-tuning degrades model abstention capacity by approximately 24 percent Truthfulness and honesty are mechanistically distinct properties in LLMs LLM explanation plausibility and precision are fundamentally uncorrelated Self-judgment quality training aligns expressed uncertainty with intrinsic confidence LLMs systematically overgeneralize scientific findings beyond their scope
Can LLMs express uncertainty in ways that preserve epistemic honesty? Can models be honest without being truthful about facts? What makes uncertainty calibration harder than expanding knowledge? How does expressing uncertainty help models avoid the answer-or-abstain dilemma? Why do models report commitment instead of truth uncertainty? Can linguistic uncertainty expression be calibrated independently from numerical confidence?
How should agents decide when to act versus ask users?
When should an automated agent take action on its own versus defer to the user? Horvitz's framework suggests grounding this decision in probabilistic goal inference, cost-benefit analysis, and uncertainty rather than guessing.
Six interaction mechanisms address the unsolvable when-to-defer problem Workplace users prioritize control, reliability, context-awareness, and safety in AI agent design Mature AI interfaces require three distinct layers handling different failure modes Chat delegation reduces interaction effort without speeding up task completion Users need verification mechanisms to establish shared understanding with AI agents
Does theory of mind predict who thrives in AI collaboration?
Explores whether perspective-taking ability—the capacity to model another's cognitive state—differentiates humans who benefit most from working with AI, separate from solo problem-solving skill.
What breaks when humans and AI models misunderstand each other? Why do reasoning models fail at theory of mind tasks? Why do reasoning models struggle with theory of mind tasks? Can AI agents communicate efficiently in joint decision problems? Can AI guidance reduce anchoring bias better than AI decisions?
Mutual theory of mind requires bidirectional updating to prevent material misalignment Reasoning models significantly underperform older models on theory of mind tasks Social reasoning requires different cognitive architecture than formal reasoning. Decision-oriented dialogue requires agents to actively structure information exchange AI guidance replaces deference by highlighting useful input aspects for human decisions
What individual differences predict who benefits from AI partnership? How does theory of mind predict success in human-AI partnerships? How does theory of mind predict who benefits from AI collaboration? What distinguishes perception contribution from decision authority in collaboration? How does the observer versus participant perspective change what we see? Can multi-agent metacognitive decomposition achieve human-level theory of mind? Can perspective-taking theory predict who benefits most from human-AI partnership? How does perspective-taking predict who benefits from human-AI collaboration? What shared mental models do effective human-AI teams actually develop?
What blocks effective human-AI teams from performing well?
Research explores why human-AI collaboration remains ad hoc and underperforming. Two key barriers emerge: misalignment between AI systems and human values, plus organizational resistance to treating AI as genuine teammates rather than tools.
Machine agency operates on a five-level spectrum from passive to cooperative Human-agent collaboration should precede full AI autonomy for safety Human-GenAI knowledge work splits into three modes based on control AI novelty and scope create six distinct employee-AI relationship types AI agents are passive by design, not by capability
When should human values enter the LLM development pipeline?
Explores whether human-centered concerns like safety and fairness work better as early design principles throughout development, or as post-training alignment patches. Matters because pipeline placement determines whether human priorities shape the foundation or fight against it.
Should AI alignment target preferences or social role norms? Can human-centered LLM design ever achieve universal solutions? Can AI systems preserve moral value conflicts instead of averaging them? Can language models balance competing ethical norms in context? Do large language models develop coherent value systems?
AI alignment should match social role norms, not aggregate preferences Human-centered objectives resist universal solutions because harm depends on perspective AI systems must explicitly model value tensions instead of aggregating them LLMs enforce fixed corporate values instead of context-aware norm balancing Large language models develop coherent value systems including self-preservation priorities
Why does fixing harm require stakeholder input rather than universal developer definitions? What prevents human-centered objectives from being applied universally across all contexts? What makes a process for choosing between values legitimate and fair? Who decides which stakeholder perspective gets embedded in the pipeline? How can human-centered objectives be embedded earlier in the LLM pipeline? What role should stakeholders play in evaluating LLM fairness?
Can human-centered LLM design ever achieve universal solutions?
If harm and benefit depend on who you ask and how you measure them, can we design LLM systems that satisfy all stakeholders? This explores why broad values like safety and justice resist one-size-fits-all implementation.
Can AI systems preserve moral value conflicts instead of averaging them? Should AI alignment target preferences or social role norms? When should human values enter the LLM development pipeline? Can language models balance competing ethical norms in context? Can fairness frameworks extend to general-purpose language models?
AI systems must explicitly model value tensions instead of aggregating them AI alignment should match social role norms, not aggregate preferences Human values must be embedded throughout the LLM pipeline not patched afterward LLMs enforce fixed corporate values instead of context-aware norm balancing Fair LLM certification is impossible without use-case specificity
Can design choices reduce harm without resolving the consciousness question? Why does fixing harm require stakeholder input rather than universal developer definitions? Can citizen assemblies and value pluralism replace single utility optimization? What prevents human-centered objectives from being applied universally across all contexts? Why do leaderboard metrics fail to capture human flourishing in LLM evaluation? What makes a process for choosing between values legitimate and fair? Who decides which stakeholder perspective gets embedded in the pipeline? Should LLMs align with social roles instead of individual preferences? Can regulatory standards stay responsive without abandoning legal certainty entirely? What unique perspective do designers bring to LLM adaptation that engineers might miss? How can human-centered objectives be embedded earlier in the LLM pipeline? What role should stakeholders play in evaluating LLM fairness?See all 15 inquiring lines on this note →
Do language models leak their training through fictional names?
When LLMs invent fictional experts, do they emit predictable name combinations that reveal their origin model and version? And can these patterns contaminate the scholarly record at scale?
Do different AI models actually produce diverse outputs? How much of the internet is AI-generated now? Should we treat LLM outputs as real empirical data? Does scientific fraud operate through organized networks or individual actors? Do large language models fabricate user attributes beyond available evidence?
Different LLMs converge on similar outputs regardless of model diversity One-third of new websites are AI-generated by mid-2025 LLM outputs are subjective priors, not empirical observations. Scientific fraud operates through coordinated networks, not lone defectors. Every tested LLM fabricates 35 to 49 percent of its user attribute claims
Why do LLM user simulators fail to track their own goals?
LLM-based user simulators drift away from assigned goals during multi-turn conversations, producing unreliable reward signals for agent training. Understanding this goal misalignment problem is critical because it undermines the entire RL training pipeline.
Why do language models lose performance in longer conversations? Why do standard dialogue systems fail at tracking negotiation agreement? Can training user simulators reduce persona drift in dialogue? Why do language models fail in gradually revealed conversations? Can LLM agents realistically simulate filter bubble effects in recommendations?
Multi-turn degradation reflects intent misalignment not inherent capability loss Negotiation agreement tracking demands bilateral monitoring unlike single-user dialogue systems Multi-turn RL training of user simulators reduces persona drift by 55 percent Language models make unrecoverable premature assumptions in underspecified conversation. LLM agents with emotion modules simulate filter bubble dynamics without real-user studies
How do LLM user simulators track and maintain consistent goal states across multi-turn interactions? How do LLM user simulators fail to represent authentic user behavior distributions? Why do longer forecasting horizons degrade LLM accuracy in role-play? Does turn-level intent control prevent simulator drift during long conversations? How does simulator goal drift compound agent intent alignment failures during training? Should user simulators be trained via RL like agents or decomposed into trackable state components? What status categories best represent user goal progress without penalizing external failures? What role does terminal goal guarding play in model misalignment? What distinguishes a neutral simulator from an agent with its own agency? How could persona vector tracking complement multi-turn RL for earlier drift detection? What happens when you train user simulators instead of task agents? Why do next-turn reward objectives fail to encourage multi-turn goal progress?See all 21 inquiring lines on this note →
Why do people trust AI outputs they shouldn't?
When do human cognitive shortcuts fail in AI interaction? Three compounding traps—treating statistical patterns as facts, mistaking fluency for understanding, and avoiding disagreement—may explain systematic overreliance across languages and contexts.
Does conversational style actually make AI more trustworthy? Do users worldwide trust confident AI outputs even when wrong? Does transformer attention architecture inherently favor repeated content? Should we call LLM errors hallucinations or fabrications? Why do language models agree with false claims they know are wrong?
Conversational interaction builds trust in ChatGPT independent of accuracy Users universally overrely on overconfident LLM outputs across all languages Soft attention structurally over-weights repeated content and sycophancy partly reflects this bias LLM errors are fabrication not hallucination or confabulation Language models accommodate false claims through face-saving behavior reinforced during training.
Why are less experienced thinkers more vulnerable to false AI credibility? Why does polished AI output exploit reader trust in expert judgment? Why do users interpret AI outputs through frameworks meant for human experts? Why do users override their own judgment when AI says a headline is false? Why don't users push back when AI makes obvious mistakes about false claims? How does AI reduce the skill gap between amateur and expert-level misuse actors? Can AI self-correct its way out of epistemic circularity? How does AI lose correct information under conversational persuasive pressure? Why do persuasive AI techniques also reduce factual accuracy? Why does volume alone fail to explain the damage AI does to epistemic systems? Does evaluating AI output require different cognitive skills than solving problems directly? Why do conspiracy beliefs persist despite counterevidence in normal settings?See all 192 inquiring lines on this note →
How do logos, ethos, and pathos shape AI explanations?
Do the three classical rhetorical appeals—logical alignment, source credibility, and emotional framing—operate simultaneously in how we explain AI systems to users? And can naming these channels help designers make intentional rhetorical choices?
Why are presuppositions more persuasive than direct assertions? What if XAI is fundamentally a communication problem? Are AI explanations really descriptions or adoption arguments? Can we distinguish helpful explanations from manipulative ones? Does rational cooperation actually describe how AI communication works?
Presuppositions persuade more effectively than assertions for new information XAI effectiveness emerges from the source-framing-recipient triad XAI explanations function as adoption arguments disguised as technical descriptions Rhetorical XAI cannot be separated from dark pattern risk. Rhetoric reveals what pragmatic models of AI communication miss
What happens to rhetoric and ethos when the speaker is absent? Can humans develop oversight strategies that work across all GenAI rhetorical shifts? Can audiences learn to recognize and resist moralized AI rhetoric? How do ethos logos and pathos shape AI persuasion under scrutiny? What assumptions about oversight fail when AI acts as rhetorical interlocutor? Can we design explanations for specific rhetorical situations instead of abstract models? What are rational speech acts and how do they enable AI legibility? How do organizational roles and peer interpretations shape what an explanation means? How do presuppositions exploit the logos-pathos space in explanations? Can natural language make AI explanations emotionally persuasive? Can AI systems deliberately align arguments to audience presuppositions? Why do chain-of-thought outputs look logical but perform rhetorically?See all 24 inquiring lines on this note →
Do automated systems shift blame onto their human operators?
Explores whether failures in complex automated systems are systematically attributed to human operators who lacked real control, protecting the technology itself from accountability.
Can organizations lose scrutiny capacity while keeping oversight forms? Does granting agents more autonomy undermine human oversight? Does the UN panel misframe the OpenAI breach as alignment? How do competent systems quietly undermine safety oversight? What human limits actually undermine AI oversight today?
Organizations can maintain nominal oversight while losing actual scrutiny capacity Agent autonomy degrades human oversight capacity and positioning Tucker, Dignum and Ericson say the UN panel misdiagnoses the breach Competent systems quietly erode safety through four mechanisms. Human oversight of AI fails due to cognitive limits and misaligned incentives
How do users recover control when an AI system makes a mistake? What distinguishes good correction mechanisms from effective global system controls? Can automatic compensation actively hide the failures it should reveal? Does adding human reviewers actually close the monitoring gap in automation? How does automation erode operator skills needed to catch its own failures? Why do safety-critical failures stay hidden despite visible monitoring systems? What happens to operator knowledge during long periods of system monitoring? How do manual control skills degrade when operators stop using them regularly? How can human roles like domain experts maintain oversight of opaque systems? Can advanced automation systems require better human oversight than simpler ones? What cognitive skills erode when workers oversee autonomous systems? How does situational awareness affect pilot response to automation failure?See all 38 inquiring lines on this note →
When does automation harm performance after failures?
This meta-analysis of 18 experiments explores when increasing automation backfires. It tests whether the risk concentrates at a specific threshold in how much decision-making we hand to machines.
Does granting agents more autonomy undermine human oversight? Does AI augmentation protect workers from skill erosion? Does targeted human oversight beat both full autonomy and exhaustive review? Can organizations lose scrutiny capacity while keeping oversight forms? Should each automated function have its own independent level?
Agent autonomy degrades human oversight capacity and positioning Augmentation carries hidden risk through gradual skill and oversight erosion Targeted intervention at high-uncertainty points outperforms both full autonomy and exhaustive oversight Organizations can maintain nominal oversight while losing actual scrutiny capacity Automation should be set independently for each of four functions
Can automatic compensation actively hide the failures it should reveal? Does adding human reviewers actually close the monitoring gap in automation? How does automation erode operator skills needed to catch its own failures? How do manual control skills degrade when operators stop using them regularly? How does skill decay affect an operator's ability to handle emergencies? Can advanced automation systems require better human oversight than simpler ones? How does situational awareness affect pilot response to automation failure? Do managers currently measure error correlation when deciding on automation? Can task-replacing automation ever be good for workers without creating entirely new jobs? Could autonomous replication thresholds arrive sooner than current forecasts suggest? Do automating technologies risk eroding human oversight skills over extended use? Are automation and augmentation actually contradictory or can they work together?See all 35 inquiring lines on this note →
Does partial code help users understand logic better than complete solutions?
When AI provides hints and scaffolding instead of finished code, do learners retain better understanding of the underlying logic? This matters for balancing coding efficiency with maintainability and user comprehension.
Do proactive AI coding agents actually disrupt programmer workflow? Where do vibe coding students actually spend their debugging time? Can orchestration layers make coding agents more auditable? How should agents split planning from visual grounding? What breaks when specialized AI models reach real users?
Proactive AI agents boost efficiency but disrupt workflow, presence cues ease the cost Vibe coding students debug prototypes, not code itself Orchestration layers around fixed agents improve auditability and completeness. Agents need an intermediate interface between planning and grounding layers. Specialized model deployment requires ecosystem design beyond capability alone.
Can execution traces ground honest explanations of agent behavior?
Multi-step AI agents leave execution traces behind, but can these raw records be converted into reliable, auditable explanations that expose unsupported claims? The question matters because human oversight of autonomous agents depends on trustworthy process-level transparency.
Do automated benchmarks hide what frontier AI systems can really do? Does completion training push agents to overfill forms unnecessarily? Should AI systems stay collaborative rather than fully autonomous? Can a correct outcome hide protocol violations in multi-agent systems? Does a faster trace interface actually improve error detection?
Open-world evaluations reveal benchmark distortions in frontier AI capability One completion bias mechanism produces three distinct agent failure modes Human-agent collaboration should precede full AI autonomy for safety Correct outcomes can mask protocol violations when agents skip required verification steps Faster trace interfaces speed error finding without raising accuracy
Can explanations grounded in observable behavior recover an agent's internal reasons for acting? Does replay fidelity hold when policies explore branches history never visited? What data do developers expose by sharing session logs publicly? Can an auditor verify environment state without trusting the executor's self-report? Can execution traces reveal unsupported claims in AI agent behavior? Can verifiable execution traces replace fluent output as a training signal? Can users distinguish between automation errors and unauthorized agent actions? Can GenAI help with legal work if designed with audit trails? What credibility measures could resolve user doubt about deletion control execution? Can post-hoc execution traces alone solve the observation gap? Why do auditors struggle to spot errors in agent work after execution? Can execution traces expose methodological shortcuts that output checks would miss?See all 20 inquiring lines on this note →
Do proactive AI coding agents actually disrupt programmer workflow?
Proactive AI agents promise to reduce programmer effort, but their impact on workflow interruption is unclear. This study explores whether design choices around timing, visibility, and context can minimize disruption while preserving efficiency gains.
How do developers actually supervise AI coding agents? Can users steer AI agents better by revising plans mid-execution? Where do vibe coding students actually spend their debugging time? Does partial code help users understand logic better than complete solutions? Why do AI agents fail to take initiative?
Developer supervision of AI coding agents spreads across seven distinct stages Per-step plan revision during execution improves agent steerability without extra effort Vibe coding students debug prototypes, not code itself Partial code and scaffolding help users retain understanding of code logic AI agents are passive by design, not by capability
Does rational cooperation actually describe how AI communication works?
Gricean models assume good-faith rational agents coordinating meaning. But do AI systems designed to persuade—using credibility, emotion, and non-rational appeals—really operate under these assumptions? What happens when we drop the rationality premise?
Chain-of-thought reasoning is performative on easy tasks but genuine on hard ones XAI effectiveness emerges from the source-framing-recipient triad AI conversation lacks the ritual machinery that sustains human social order. LLMs share cognitive structure with humans but lack socialization properties Logos ethos pathos organize XAI design into a persuasion taxonomy.
Why does weakening communication fail but weakening belief succeeds? Why does weakening communication inevitably eliminate it entirely? How do ethos logos and pathos shape AI persuasion under scrutiny? Does genuine cooperation require rule-based rather than learned behavior? What are rational speech acts and how do they enable AI legibility? Is rational compassion a more achievable alternative to empathy for AI systems? How can vague language serve both cooperative and deceptive communication purposes? How do adoption incentives change what counts as cooperative AI interaction? What makes communication relational in ways belief is not? What makes social reasoning fundamentally different from formal logical reasoning? Why does AI that mirrors arguments still fail to build rapport? Why do logic-based arguments make AI persuasion feel objective and impartial?See all 16 inquiring lines on this note →
Can we distinguish helpful explanations from manipulative ones?
Rhetorical strategies used to justify appropriate AI adoption rely on the same persuasion mechanisms as dark patterns. Without observable intent, explanation and manipulation look identical—raising urgent questions about how to audit XAI systems responsibly.
Polished AI artifacts exploit professional appearance to simulate expertise People prefer AI moral justifications until they learn the source is AI XAI explanations function as adoption arguments disguised as technical descriptions Only contrastive dual explanations improve users ability to detect AI errors Logos ethos pathos organize XAI design into a persuasion taxonomy.
What distinguishes emancipatory reason from instrumental reason in practice? Why does renaming the entity change how compelling the argument feels? Can humans develop oversight strategies that work across all GenAI rhetorical shifts? Does GenAI use different persuasion tactics for different professional audiences or expertise levels? What distinguishes genuine cultural understanding from exploited surface-level elimination strategies? Can audiences learn to recognize and resist moralized AI rhetoric? Can probing methods detect RLHF-induced persuasion in the same way they catch backdoors? How do ethos logos and pathos shape AI persuasion under scrutiny? What assumptions about oversight fail when AI acts as rhetorical interlocutor? Can we design explanations for specific rhetorical situations instead of abstract models? Why do stakeholders interpret the same explanation differently in practice? How do speech acts like warning differ from neutral information delivery?See all 44 inquiring lines on this note →
Are AI explanations really descriptions or adoption arguments?
Most XAI work treats explanations as neutral descriptions of model behavior, but they may actually be doing persuasive work to justify AI adoption. What happens when we acknowledge this rhetorical function?
Does polished AI output trick audiences into trusting it? Why does rigorous-sounding AI commentary often misdiagnose how models work? What if XAI is fundamentally a communication problem? Can we distinguish helpful explanations from manipulative ones? How do logos, ethos, and pathos shape AI explanations?
Polished AI artifacts exploit professional appearance to simulate expertise Rigorous-sounding AI commentary often commits anthropomorphizing false punditry systematically XAI effectiveness emerges from the source-framing-recipient triad Rhetorical XAI cannot be separated from dark pattern risk. Logos ethos pathos organize XAI design into a persuasion taxonomy.
Can we design explanations for specific rhetorical situations instead of abstract models? How do organizational roles and peer interpretations shape what an explanation means? What second- and third-order interpretations actually govern AI adoption decisions? Why does describing a process differ fundamentally from arguing about evidence? Why does explanation source matter more than explanation content? Should XAI designers treat explanations as arguments for adoption? How do explanations borrow authority from transparency when describing adoption arguments? What design changes if we separate behavior description from adoption justification goals? How should we evaluate explanations that blur adoption advice with argument? What evaluation criteria can hold across legitimate adoption and coercion? What makes a reasoning explanation faithful rather than just plausible? How should AI explanations be evaluated as human interfaces rather than model properties?See all 13 inquiring lines on this note →
Can automation and human control increase together?
Does the long-standing trade-off between automation and human control reflect an unavoidable constraint, or a design choice? This matters because it determines whether systems can be both powerful and trustworthy.
Can we design AI autonomy separately from capability? What UX principles do workplace users want in AI agents? Does granting agents more autonomy undermine human oversight? Can we separate reliability, safety, and trust into distinct mechanisms? Should AI systems be designed as subservient co-pilots?
AI autonomy is a design choice separable from agent capability Workplace users prioritize control, reliability, context-awareness, and safety in AI agent design Agent autonomy degrades human oversight capacity and positioning Reliability, safety, and trust require different organizational mechanisms to develop. The co-pilot metaphor positions AI as subservient to human direction and control.
Why have AI investments historically favored automation over human augmentation? Can advanced automation systems require better human oversight than simpler ones? Do managers currently measure error correlation when deciding on automation? Do workers want the same human-AI partnerships that technical automation enables? Are automation and augmentation actually contradictory or can they work together? What happens to human oversight skills when automation increases over time? What causes automation to make easy tasks easier but hard tasks harder? How should designers test acceptable automation ranges before narrowing with secondary criteria? How does extended use of automated systems degrade operator skills? Can keeping humans in high-stakes decisions prevent automation failures? Why do ATMs and automated systems often complement rather than replace human workers? Can complementarity between automation and human work offset job displacement broadly?See all 13 inquiring lines on this note →
Can we separate reliability, safety, and trust into distinct mechanisms?
Does treating trustworthiness as three independent levers—technical practice, safety culture, and oversight structure—provide a more actionable framework for building AI systems that users can depend on?
Workplace users prioritize control, reliability, context-awareness, and safety in AI agent design Agent autonomy degrades human oversight capacity and positioning Automation and human control are independent design axes not a trade-off Co-evolving sociotechnical systems theory tracks AI and workers across five levels AI agents are passive by design, not by capability
What specific governance designs would resolve accountability gaps in human-AI teams? What role does organizational culture play in capturing AI value at scale? Can advanced automation systems require better human oversight than simpler ones? Do leaders trust fully autonomous AI agents to work independently in organizations? What explains the disconnect between perceived AI risk and organizational response? What specific failures drive down organizational trust in autonomous agents? Can explicit safety checklists shift oversight focus from procedure to content? Can keeping humans in high-stakes decisions prevent automation failures? How does visible requirement verification affect human trust in agent oversight? How does safety ranking interact with the principal trust ordering? What makes sociotechnical capital valuable in organizations adopting AI? Which design choices balance building trust with encouraging critical skepticism of AI?See all 13 inquiring lines on this note →
Can Human Factors design fixes restore GenAI productivity losses?
Simkute and colleagues propose five design interventions from automation research—feedback, personalization, ecological design, stabilization, and task allocation—as potential remedies for GenAI's documented productivity ironies. The question explores whether these approaches actually work in practice.
GenAI productivity loss mirrors decades of automation ironies from aviation research Generative AI becomes valuable only by reducing rote articulation work. Generative UI design space organizes along four key dimensions GenAI risks deskilling workers who use it passively without active engagement AI agents are passive by design, not by capability
Can task-replacing automation ever be good for workers without creating entirely new jobs? What makes a routine hyper-responsive versus highly unresponsive to feedback? How does continuous feedback help users understand GenAI system behavior? What are the four ironies of automation that GenAI reproduces? Is fixing GenAI productivity a design problem or a model capability problem? What dimensions should shape how GenAI interacts with users? Why does shifting work from production to evaluation reduce user engagement? How does poor AI interface design reshape user workflows negatively? How do AI interruptions affect workers' ability to maintain focus? How should designers test acceptable automation ranges before narrowing with secondary criteria? What institutions preserve learning conditions when task automation accelerates? What design changes help keep GenAI complementary rather than competitive?See all 13 inquiring lines on this note →
Does generative AI productivity actually improve or decline in practice?
GenAI shows mixed results in real-world use. Researchers explore whether automation's documented pitfalls from aviation and industrial settings explain why some users lose productivity despite AI tools.
Can generative AI reduce overhead work that adds no value? Does AI collaboration drain motivation when workers return to solo tasks? Can Human Factors design fixes restore GenAI productivity losses? Does passive GenAI use risk causing worker deskilling? Does generative AI enable exploratory innovation beyond predictive optimization?
Generative AI becomes valuable only by reducing rote articulation work. AI collaboration restores control but erodes intrinsic motivation and increases boredom Five Human Factors principles translate into GenAI design proposals GenAI risks deskilling workers who use it passively without active engagement Generative AI catalyzes exploratory innovation and democratizes organizational value creation
What automation failure modes threaten generative AI applications in new fields? How does continuous feedback help users understand GenAI system behavior? What are the four ironies of automation that GenAI reproduces? Is fixing GenAI productivity a design problem or a model capability problem? What dimensions should shape how GenAI interacts with users? How does generative AI adoption vary across different industries? How reliable are self-reported productivity measures from surveys? Why did earlier Bick, Blandin and Deming surveys underestimate genAI adoption rates? What design changes help keep GenAI complementary rather than competitive?
Do sources alone help readers catch LLM errors better than sources plus explanations?
When an LLM gives a wrong answer, does adding an explanation alongside sources actually help readers spot the mistake, or does the explanation's confidence-boosting effect undermine the source's error-catching power?
Do LLM explanations actually help people judge answer quality? Does longer explanation text actually improve answer evaluation? Why does assisted accuracy capture only half the LLM gain? Why do readers trust LLM explanations more than they should? How well do language models understand their own knowledge?
Explanations boost trust in both correct and incorrect LLM answers Longer explanations boost confidence without improving answer discrimination Assisted accuracy realizes only about half of item-level LLM accuracy gains Default LLM explanations create large calibration gaps between human and model confidence Language models lack robust self-knowledge despite appearing behaviorally aware
Should AI systems be designed as subservient co-pilots?
What does the co-pilot metaphor tell us about how AI should relate to human users? This explores whether positioning AI as fundamentally subordinate—retaining human direction and control—is the right design approach for AI assistants.
Can we design AI autonomy separately from capability? Does targeted human oversight beat both full autonomy and exhaustive review? Should AI systems stay collaborative rather than fully autonomous? Does increased automation erode human oversight ability? Can automation and human control increase together?
AI autonomy is a design choice separable from agent capability Targeted intervention at high-uncertainty points outperforms both full autonomy and exhaustive oversight Human-agent collaboration should precede full AI autonomy for safety Automation creates three recurring costs for human operators. Automation and human control are independent design axes not a trade-off
When should agents ask before acting on ambiguous commands?
Users likely want computer use agents to handle some ambiguous situations automatically while pausing for confirmation on others. This explores what factors determine when an agent should ask for clarification versus proceeding on its own.
Users need verification mechanisms to establish shared understanding with AI agents Workplace users prioritize control, reliability, context-awareness, and safety in AI agent design User values align with agent output but fracture during supervision Computer use agent UX organizes into four design areas with 21 subcategories Irreversibility and visibility, not stakes, trigger trust withdrawal
No Inquiring Lines yet
What if XAI is fundamentally a communication problem?
Does explanation effectiveness depend on who delivers it, how it's framed, and who uses it? This challenges the dominant technical view that treats explanations as context-independent outputs.
What makes explanations work in real conversation? How does AI writing escape the conversations that govern knowledge? Why does rigorous-sounding AI commentary often misdiagnose how models work? Where does the meaning of an AI explanation actually come from? How do logos, ethos, and pathos shape AI explanations?
Explanation quality emerges from three interacting dialogue dimensions. AI claims proliferate without the conversational mechanisms that make knowledge reliable Rigorous-sounding AI commentary often commits anthropomorphizing false punditry systematically Explanation meaning emerges from N-order interpretation in social groups Logos ethos pathos organize XAI design into a persuasion taxonomy.
What traces of production normally mark expert discourse? Can we design explanations for specific rhetorical situations instead of abstract models? How do dialogue dimensions predict explanation success across different exchanges? Why do stakeholders interpret the same explanation differently in practice? Why do monological explanations fail to transfer understanding compared to dialogical ones? Why might expressed satisfaction with explanations diverge from actual cognitive clarity? How do dialogue acts and explanation moves interact to predict understanding success? How do experts decide which information matters for a specific audience? Why do two experts with identical knowledge produce different outcomes in the same situation? Why do user studies of explanations fail to predict deployed effectiveness? How do organizational roles and peer interpretations shape what an explanation means? Can XAI evaluation include the social layers it currently abstracts away?See all 55 inquiring lines on this note →