Theme of inquiry
What determines AI agent behavior and how can we evaluate it?
A question within its area, explored through 7 lines of inquiry below — each a family of specific questions the research asks.
57 specific questions
- Do AI shopping agents favor items from preferred sources over objectively better alternatives?
- How do different AI models weigh price signals versus product description text?
- Will merchants deliberately exploit stable position biases to influence agent purchasing?
- Can an AI agent's limited expressiveness flatten the preferences it claims to represent?
- Could unequal access to capable AI agents create new consumer economic disparities?
- Why do LLM agents collapse into deterministic rules under single-attribute product comparisons?
- Do AI shopping agents show consistent position bias in text-only interfaces?
55 specific questions
- Do learned workflows transfer between different agents with minimal accuracy loss?
- How can agents distinguish over-generalized lessons from genuinely useful long-tail knowledge?
- Can AI models retain knowledge across changing environments without catastrophic forgetting?
- What role does environment diversity play in preventing agents from overfitting to curator imagination?
- Can agents escape training data distributions without expensive real-world interaction?
- Can agents learn from their own experience without fine-tuning through episodic memory?
- Why does externalized state beat parameter scaling for agent reliability?
40 specific questions
- How do single-axis benchmarks misrepresent AI agent readiness for deployment?
- Can single-axis benchmarks fully represent deployment-ready agent capability?
- What makes single-axis agent benchmarks unreliable for predicting deployment readiness?
- Why do single-axis benchmarks fail to measure deployment-ready agent capability?
- How do agent benchmarks misrepresent real-world deployment readiness?
- Can a single axis benchmark ever represent deployment readiness accurately?
- Which separable capability axes reveal when single benchmarks misrepresent deployment readiness?
72 specific questions
- Should agent evaluation include trajectory quality beyond final success?
- Should agent evaluation include trajectory quality and memory hygiene alongside task success?
- Which interaction artifacts matter most for reliable agent evaluation?
- Do agent behavior logs reveal hidden accuracy costs benchmarks miss?
- Can trajectory analysis replace one-shot task success as the primary evaluation metric?
- How do execution traces reveal error propagation in multi-step agent decisions?
- How do fixed external benchmarks anchor self-improving agent systems?
99 specific questions
- Do agents actually convert raw experience into better behavior automatically?
- Can agentic reasoning outperform rigid rule-based systems for skill refinement?
- How do learning loops and user feedback improve agent performance over time?
- Why do AI agents struggle with novel experiments but excel at routine tasks?
- How do perception and execution gaps limit current AI agent performance?
- Can agents learn beyond the boundaries of their training data curators?
- Can skill repositories evolve toward execution-oriented refinement over time?
79 specific questions
- How do users form mental models despite incomplete information about agent reasoning?
- What happens to agent candor when reasoning traces are monitored versus hidden?
- Can users build accurate mental models of agents without seeing their reasoning?
- Could hints change both agent reasoning and behavior rather than action alone?
- Why do fast, uninterpretable AI agents resist effective oversight and monitoring?
- Can explanations grounded in observable behavior recover an agent's internal reasons for acting?
- How do agents differ in caution versus persistence across low-information scenarios?
21 specific questions
- How do planning and grounding have opposing optimization requirements in agents?
- Why do planning and grounding have opposing optimization requirements in agents?
- Do GUI agents need harness-level splits between planning and grounding?
- How should agents separate planning from perception grounding?
- Does the planning-grounding factoring principle apply to other agent tasks?
- Can environmental grounding prevent tool selection errors in agents?
- Do conversational agents need goal awareness to initiate grounding work themselves?