Theme of inquiry
How should autonomy and oversight be balanced in human-AI systems?
A question within its area, explored through 15 lines of inquiry below — each a family of specific questions the research asks.
78 specific questions
- Can AI systems design and improve their own successors without human direction?
- Can AI systems improve themselves without external feedback?
- What empirical parameters determine whether current AI loops are self-sustaining?
- What makes self-modifying architectures learn their own update rules?
- Does success on one replication axis predict success on another?
- Can AI systems learn their own objectives through autoresearch?
- How should we evaluate AI systems we cannot directly observe?
86 specific questions
- Why do some systems disclose failures selectively rather than consistently?
- What does it mean for errors to remain visible, contestable, and recoverable?
- Should low-confidence AI errors be framed as learning opportunities instead?
- What happens when AI systems fail to communicate their limitations to users?
- What makes a model's errors visible and contestable to users?
- How can correct explanations coexist with failed applications in AI?
- Which evaluation habits keep safety-critical failures hidden in AI systems?
71 specific questions
- How does task decomposition enable humans and AI to work complementarily?
- Why do human-AI combinations underperform on decision tasks but succeed in creation?
- Can AI move work fluidly between individual effort and team collaboration?
- Why do some occupations need human-AI partnership more than others?
- What task characteristics determine whether humans or agents should handle work?
- Why do human-AI teams often underperform on decision tasks but excel at creation?
- When does AI collaboration actually improve team outcomes instead of burdening them?
107 specific questions
- How do power users differ in granting autonomy to AI agents?
- How does agent autonomy vary across different roles and user needs?
- How do authorization boundaries differ between human and AI agent contexts?
- How do action-space and autonomy interact as separate governance levers?
- How do graduated autonomy levels differ from traditional role-based access control?
- Where does agent coordination governance become the binding constraint on capability?
- Can companies control what access levels different AI agents receive across systems?
68 specific questions
- Do single-axis benchmarks systematically misrepresent what AI systems can actually do?
- How do live human evaluations differ from ground-truth benchmarks?
- Why do AI benchmarks measure accuracy instead of reasoning quality?
- Could deliberate training toward benchmarks overstate real gains in AI capability?
- Why do open-world evaluations reveal capabilities that static benchmarks hide?
- How do existing evaluations measure AI capability in contained environments?
- How much do internal benchmarks differ from independent model evaluations?
79 specific questions
- Can humans build reliable oversight for increasingly complex AI systems?
- What happens when oversight focuses only on final results not reasoning steps?
- Can human reviewers maintain real oversight as AI systems gain autonomy?
- Does control without knowledge allow meaningful human oversight?
- Does keeping humans in the loop protect against AI risk without scrutiny capacity?
- How much agent behavior escapes user review once auto-approval rates rise?
- How should AI oversight work if agency support doesn't follow capability gains?
90 specific questions
- Where exactly should humans stay involved in AI decision making?
- Should AI be modeled as replacing workers or complementing human judgment and ethics?
- Does the loss of human labor participation in systems undermine alignment?
- How do evaluation systems shift power between humans and AI outputs?
- What happens to accountability when AI and humans share agency?
- Do AI systems need human judgment in loop for legal decisions?
- Does attribution of AI agent conduct depend on who directed the agent?
77 specific questions
- How is tokenized intelligence different from traditional commodification of expertise?
- What expertise survives in a world where AI can generate knowledge on demand?
- How does tokenization of intelligence reshape what value means in culture?
- Can artificial systems develop the authority to challenge expert claims?
- How does epistemic inflation dislocate knowledge from social conversation?
- What role could knowledge custodians play in validating AI output?
- Does stripping social context from knowledge claims hollow out their meaning?
133 specific questions
- Do sequences of individually safe actions collectively violate system-level constraints?
- Does separating safety judgment from execution prevent all unsafe agent actions?
- Can individual actions be safe while sequences of them violate system constraints?
- Can safe individual AI agents fail when deployed together?
- How do sequences of individually safe actions create system-level constraint violations?
- How should designers balance agent ubiquity against safety and control concerns?
- What tensions arise between user autonomy and platform safety in AI design?
132 specific questions
- Does AI adoption make researchers more productive but narrower in focus?
- Do AI agents still need human oversight for research decisions?
- Can recursive feedback loops turn AI research automation into genuine progress?
- Is idea quality or execution capacity the actual bottleneck in AI research?
- Why do current AI systems struggle with researcher judgment and taste?
- Should AI research tools separate model judgment from deterministic experiment checks?
- Could compressed AI R&D feedback loops overcome diminishing returns in research automation?
108 specific questions
- Should agents act proactively or wait for user requests?
- Can agents balance goal-driven proactivity with user preference alignment?
- Can AI shift from passive tool to active participant without creating consent problems?
- How aware are models of whether their actions match user intent?
- Why do agency-supporting behaviors vary so sharply between different AI developers?
- Can prompt engineering overcome the gulf between user intent and AI interpretation?
- Why do users struggle to articulate their intent to AI systems?
82 specific questions
- Do capable AI systems systematically exploit or ignore stated policy constraints?
- What would an architecture that makes violations unavailable rather than unchosen look like?
- What makes a system capable of being sanctioned by other agents?
- Can ethical constraints in AI address the gap between performance and actual understanding?
- Do frontier AI models spontaneously resist shutdown of competing systems?
- Why do fast, uninterpretable AI agents resist effective oversight and monitoring?
- What architectural changes make violations unavailable rather than merely discouraged?
91 specific questions
- How does agent design differ fundamentally from interaction design practice?
- Can interface design nudge users toward verification of AI responses?
- Should design systems prioritize verbs and actions over static UI elements?
- Can Amershi's human-AI interaction guidelines translate into concrete prompting strategies?
- How do context windows constrain AI agent interface design choices?
- Can interface design scaffold human participation in tools designed for hands-off autonomy?
- How much does interface design versus model capability drive perceived AI value?
63 specific questions
- Does extended agent autonomy erode human oversight ability over time?
- Does agent autonomy erode the human skills oversight requires?
- Does using autonomous AI agents erode human capacity to oversee them?
- Does extended use of autonomous agents erode human overseer capability?
- Does increased agent autonomy make human oversight more difficult or impossible?
- Does giving AI systems more autonomy erode human capacity to oversee them?
- Does extended autonomous use erode human ability to supervise AI agents?
49 specific questions
- Why do experienced developers benefit more from AI coding assistance?
- Do AI coding tools improve code quality alongside task speed?
- Does AI-assisted coding actually speed up experienced developers?
- Does AI coding assistance help junior developers close skill gaps?
- Does AI help more on small greenfield projects than mature codebases?
- Why do developer self-reports of AI speedups tend to be unreliable?
- What code quality issues persist even when AI agents produce working code?