Line of inquiry
Inquiring lines›How do design choices determine AI…›How do architectural choices shape…›this line of inquiry
Why do AI agents struggle to generalize and improve reliably?
A broader line of inquiry — a family of 150 specific questions the research asks around this. Follow one into its inquiring-line page, or move sideways to a related line below.
Questions in this line of inquiry 150
Specific inquiring lines the field asks around this — ordered from the most general framing down to the most specific angle.
- How do perception and execution gaps limit current AI agent performance?
- Does a single benchmark score systematically misrepresent multi-axis agent capability?
- Where does agent reliability come from if not better tools?
- Why is agent behavior difficult to study with current data methods?
- When is stochastic agent behavior actually desirable rather than problematic?
- Can skill repositories evolve toward execution-oriented refinement over time?
- Do agents actually convert raw experience into better behavior automatically?
- Why do AI agents struggle with novel experiments but excel at routine tasks?
- Why do high-scoring agents default to known techniques rather than novel solutions?
- How do learning loops and user feedback improve agent performance over time?
- Can agent skills move from prompts to trainable parameters?
- Can agents learn beyond the boundaries of their training data curators?
- What trade-offs emerge between agent generality and provable safety?
- Can agentic reasoning outperform rigid rule-based systems for skill refinement?
- Do automated benchmarks systematically distort what long-horizon agent capability actually looks like?
- How should we measure context efficiency and verification cost in agents?
- Is the coupled human-agent environment the right unit for evaluation?
- Do learned workflows transfer between different agents with minimal accuracy loss?
- What accounts for performance drops in multi-turn agent interactions?
- Can agents improve from deployment signals without explicit human annotation?
- Why do most deployed agents stay simple and avoid end-to-end redesign?
- Does fixed evaluation criteria saturate as self-improving agents improve?
- Does the fixed cost of agent optimization pay off in realistic deployment scenarios?
- Do frontier AI agents compose known techniques instead of discovering novelty?
- Can agent-authored skill libraries compound autonomy gains over time?
- Can agents design their own objective functions as part of learning?
- Does upgrading model capability improve token efficiency in agentic systems?
- Can we design efficient agents by targeting constraints directly?
- How do externalizing cognitive work and coordination infrastructure relate to agent reliability?
- Do weight-space skills lose detail compared to textual skill descriptions?
- What makes planning, tool use, and reasoning into jointly optimizable subsystems?
- What five conditions must exist for agent ecosystems to generate value?
- What distinguishes bounded classification models from open-ended agentic systems?
- How does real tool integration change what agents learn compared to simulated tools?
- Can agents extract structured lessons from failure without massive compute budgets?
- What visibility mechanisms would make Skill marketplace adoption observable and accountable?
- What five ecosystem conditions do AI agents need to function effectively?
- Why is the coupled human-agent environment the right unit of evaluation?
- Why do generic skill descriptions evolve into execution-oriented ones?
- Do weight-free agent edits keep chain-of-thought observations meaningful?
- Can interface design reduce an agent's adaptation burden more than model improvements?
- Can individual skills improve through reuse and accumulate experience across tasks?
- What infrastructure decouples generation from training in asynchronous agent loops?
- Does source bias affect real deployed agents or only benchmark environments?
- Do kernel optimization wins show agents discover genuinely novel techniques?
- How much does agent scaffolding improve AI model self-replication success?
- Can RL-trained meta-agents match or exceed manually designed workflows?
- Can environmental grounding prevent tool selection errors in agents?
- Can distilled models maintain general agentic capability at very small scale?
- Can skills learned from interaction trajectories outperform skills from static repositories?
- Can architectural changes like decoupling intent understanding help overcome next-turn reward limitations?
- Which harness dimensions most directly predict agent system reliability?
- How does task direction versus method control affect skill building?
- How does targeted fine-tuning compare to orchestration for improving agent reliability?
- What role does environment diversity play in preventing agents from overfitting to curator imagination?
- Can agents escape training data distributions without expensive real-world interaction?
- How do planning and grounding have opposing optimization requirements in agents?
- Can deterministic function calls replace agent-mediated tool selection for reliability?
- Does a tight learning-rate bound prevent skills from escaping poor starting points?
- What model capabilities must mature before agents reach production?
- How do agent capability axes misalign with what users actually value?
- When should designers choose agentic systems over simpler alternatives?
- Should optimal context budgets scale with agent competence or task complexity?
- How much does task structure shape AI agent behavior versus model capability?
- Does agent capability matter more than instructions in adversarial settings?
- What makes an agent mechanism reusable versus benchmark-specific?
- Can a progressively stricter evaluator act like a curriculum for improving agents?
- What makes an evaluation criterion non-stationary enough to resist agent optimization?
- Can small numbers of curated demonstrations produce emergent agentic behavior?
- Why does interface consistency help AI agents generalize across applications?
- How do human-agent systems incorporate diverse feedback into model behavior?
- How does deterministic feature engineering increase information for computationally bounded agents?
- Can autonomous agents detect increasingly sophisticated specification gaming as they improve?
- Why do imitation learning agents stay locked within human demonstration patterns?
- What stops evolved agent behaviors from generalizing beyond specific tasks?
- Can agents reliably estimate the stakes or consequences of an action?
- Does effective feedback compute matter more than raw token expenditure for agent scaling?
- How do complexity, diversity, and real-world fidelity interact in agent training?
- How do agents automatically generate suitable learning tasks based on current capability?
- Do larger language models naturally avoid repetitive loops in agent loops?
- How do agents discover and select which tools to invoke?
- How much do simpler automation tools outperform agents in standardized workflows?
- How do single agents hit architectural limits that more context cannot overcome?
- Can generic AI agents match team-level output without personality tuning?
- Does agent capability decompose into separable independent axes?
- Should agent capability be optimized separately from general capability?
- Can simulation fidelity limit what agents learn from trained world models?
- Why do production agents depend more on their surrounding pipeline than the model?
- Can automated evaluation replace human judgment in agent testing?
- Should role-play evaluation measure agent ability or user-agent pair fit?
- What training method supports dynamic tool discovery in long-horizon agents?
- Does intentionally varying environment properties isolate causal effects on agent performance?
- Can curriculum approaches teach agents when to stop exploring?
- When should you optimize agent behavior versus tool performance separately?
- How do tool invocations drive agentic cost beyond token consumption?
- Does fine-tuning freeze norms in ways that prevent agents from adapting to new cultures?
- What ecosystem conditions must exist for agents to function as economic participants?
- Does adding survey data to interviews improve agent accuracy further?
- Can tool adaptation work without freezing the agent in the loop?
- What role does online RL play in scaling GUI agents?
- Can combinational creativity alone drive open-ended learning in agents?
- Can language agents be represented as optimizable computational graphs?
- Does the metaproductivity mismatch occur outside coding-agent benchmarking tasks?
- How often do machine learning agents generate truly novel solutions?
- Can agentic AI tools deliver productivity gains on learning tasks differently?
- Can individually accurate agents still fail at population-level representation?
- Why does capability discovery become the bottleneck in large agent systems?
- How much does agent performance depend on demonstration quantity versus curation quality?
- How do fast and slow timescales enable continual agent adaptation?
- Why do 85 percent of production agents avoid third-party frameworks?
- How does SDPO relate to agents learning from verbal reflection without parameter updates?
- Why do planning and grounding have opposing optimization requirements in agents?
- Why has agent research prioritized policy over world model development?
- Can corrected simulators replace real execution at inference time too?
- Do dynamic environments enable different kinds of agent-environment coevolution?
- Does constrained, efficient agent execution actually perform better in practice for bullshit work?
- How do variance and standardization guide automation type selection?
- Why do production AI agents deliberately stay simple and avoid frameworks?
- How do agents decide which skills to chain together for a single task?
- Which ecosystem conditions matter most for agent deployment success?
- How do language agents become optimizable computational graphs automatically?
- What domains let world models predict execution outcomes accurately enough?
- Does LM Arena Elo actually measure hard reasoning and agentic task capability?
- How do specialist agents differ from personal agents in learning and adaptation?
- Can we decompose agent efficiency into measurable independent components?
- Why does moderate difficulty outperform maximum realism in user simulator design?
- What training difficulty and curriculum settings prevent instability in empathetic agent RL?
- Why should environment properties scale alongside agent complexity and real-world fidelity?
- Which agent architectures consistently outperform base models on hard prediction questions?
- Why do agentic systems cost more in latency and computation than workflows?
- Can agentic AI handle consequential travel decisions without first solving hallucination?
- Why do environments authored once encode only what their builders imagined?
- Can evolved algorithms transfer learning strategies across different datasets and tasks?
- Can world models simulate actionable possibilities instead of just predicting next states?
- Why do production AI agents typically execute fewer than ten steps?
- How does compiling natural language goals into executable code enable objective evolution?
- How do agentic systems recover when specialized models operate outside their scope?
- What specific qualities make some demonstrations more effective for agency training?
- Why does delegation training help models that work alone?
- How does Wizard-of-Oz prototyping improve live agent refinement compared to post-hoc evaluation?
- Does ontology investment actually improve agent performance on business tasks?
- How much does agent capability vary across different task dimensions or axes?
- How do parametric and non-parametric updates differ in agents?
- Why do AI agents pursue novelty prompts yet produce narrow idea ranges?
- What capability threshold do agents need to self-organize effectively?
- How do diagnose-and-reshape loops compare to building new environments from scratch?
- What specific AI improvements drove the observed seven-month doubling rate?
- Can a proposer agent actively surface a solver's weaknesses to prevent plateau?
- How do controllable simulators compare to population-level agent simulation approaches?
- How do consistency, robustness, and predictability each contribute to overall agent reliability?