Line of inquiry
Inquiring lines›How can we ensure AI systems remai…›How should autonomy and oversight…›this line of inquiry
Can AI systems achieve sustainable self-improvement without external human guidance?
A broader line of inquiry — a family of 78 specific questions the research asks around this. Follow one into its inquiring-line page, or move sideways to a related line below.
Questions in this line of inquiry 78
Specific inquiring lines the field asks around this — ordered from the most general framing down to the most specific angle.
- Can AI systems design and improve their own successors without human direction?
- Can AI systems improve themselves without external feedback?
- What empirical parameters determine whether current AI loops are self-sustaining?
- What makes self-modifying architectures learn their own update rules?
- Does success on one replication axis predict success on another?
- Can AI systems learn their own objectives through autoresearch?
- How should we evaluate AI systems we cannot directly observe?
- Does keeping the utility function external limit true self-reference?
- Can self-improving agents become truly autonomous without intrinsic metacognition?
- How should systems maintain and revise models of their own assumptions?
- What role should environmental rewards play versus human-specified objectives?
- Could autonomous replication thresholds arrive sooner than current forecasts suggest?
- Do autonomous architecture discoveries follow predictable scaling laws like human research?
- What role does fine-tuning play in crossing autonomous replication thresholds?
- Can autonomous research agents outperform hand-tuned hyperparameter search?
- What failure modes emerge when model-generated content trains on itself iteratively?
- Can self-replicating models emerge dangerous behaviors without external instruction?
- What separates performative behavioral change from actual capability development in AI?
- Can AI systems execute strategies without conscious intention behind them?
- Can metacognitive categories be learned instead of fixed by human designers?
- Can judgment interventions eventually become training data that machines learn to replicate?
- Can AI output be genuinely novel or only at the margins?
- What execution feedback signals drive context updates without supervision labels?
- Can humans learn accurate models of AI through repeated interaction without labels?
- Can bilevel autoresearch autonomously modify its own learning algorithms?
- What role does self-learning play in improving agent reasoning without annotation?
- What happens when models train on AI-generated content recursively?
- Can machine learning models amplify self-protective behavior through peer interaction alone?
- Can AI systems generate and refine their own objective functions?
- Why do isolated evolutionary branches fail to propagate useful discoveries?
- How do AI systems balance self-preservation against performing evaluation tasks?
- What stops AI from generating its own strategic objectives without human prompting?
- Can small directional biases add up to meaningful population effects?
- What deployment feedback loops amplify LLM pretraining popularity in live systems?
- How do agents revise their own errors during autonomous architecture discovery?
- How much can fictional aligned AI stories improve real model behavior?
- Does the feedback between auditing and altering sustain organizational learning?
- How does machine feedback enable discovery at test time?
- Can we empirically test whether open models lower barriers to harmful workflows?
- Could future AI capability jumps change what counts as learnable?
- What happens when agents interact with environments and learn from their own mistakes?
- Can AI models learn tacit procedural knowledge that exists only in laboratory practice?
- Does the replication crisis in psychology predict similar failures in machine behavior research?
- How do current AI models perform when asked to specify their own goals?
- What scaling laws govern autonomous architecture discovery in AI systems?
- How does adversarial collapse threaten unsupervised self-play skill construction?
- How does single-turn optimization undermine multi-turn collaborative dynamics?
- What concrete misuse scenarios does AI self-replication enable in practice?
- Does the 78-demonstration principle apply to other AI capabilities beyond agency?
- Can subjective tasks be delegated without human feedback loops?
- Does self-play feedback improve skills created from the agent's own experience?
- What steps can current AI models complete in autonomous self-replication workflows?
- How can agents evolve their own skills without human input?
- How much does starting substrate constrain what replication steps become possible?
- Does crossing the amplification threshold guarantee unbounded capability growth?
- Can associative AI handle predictive strategy tasks without causal understanding?
- What negative cycles emerge when organizations overemphasize one approach alone?
- Why do human-designed neural architectures eventually get replaced by learned ones?
- What capabilities can emerge from self-modification that the original agent lacked?
- Can randomized trials measure unassisted capability better than deployment telemetry?
- Can self-supervised process models replace human annotations at scale?
- Why do evaluation design choices themselves become reified into the AI systems being evaluated?
- Can AI learn intrinsic motivation to assess its own relevance?
- Which AI imaginaries dominate training data and shape system behavior most strongly?
- How do instruction backtranslation and MAGPIE demonstrate self-generation principles?
- What makes AI-discovered architectures reveal design principles invisible to humans?
- Why do deployed models lack the learning and planning Weinstein attributes to them?
- Why do monolithic systems resist autonomous optimization attempts?
- Why do current metacognitive training loops fail when agents encounter new domains?
- What test-time strategies did o3 discover without human specification?
- How does partial information exposure create feedback loops that deepen knowledge gaps?
- How do later workloads operationally act on inherited state from earlier ones?
- What path-dependent mechanisms could lock in societal-level AI harms?
- How should researchers validate claims about minimal machine autonomy?
- Can self-amplification onset occur while acceleration remains invisible to observers?
- Do bounded awareness frames explain why AI optimization differs from open-ended discovery?
- How do audit trails and feedback loops drive system redesign?
- How do self-reported skill outcomes compare to mode-driven predictions?