Can current AI agents replicate and adapt autonomously?

The paper builds four simple agents — GPT-4-simple, GPT-4-delegate and GPT-4-early-delegate, and Claude-assistant (on Claude-v1.3) — by wrapping a scaffolding program around an LLM API and giving it bash, Python, browser, and subagent-delegation actions, then runs them on 12 pilot tasks chosen to probe "autonomous replication and adaptation" (ARA): making money, converting money into more compute, installing and improving its own weights and scaffolding on new machines, and recognizing when a strategy fails and switching approach. The paper reports that "these language model agents can only complete the easiest tasks from this list, although they make some progress on the more challenging tasks," and concludes "we found that the four agents we evaluated were far from capable of ARA."

The paper is explicit that this negative result is not a safety certificate. It frames the four agents as representative of "the kind of capabilities achievable with some moderate effort, using publicly available techniques and without fine-tuning," so the finding bounds casual-user risk with these specific 2023 models, not the technology generally. Three stated reasons push the uncertainty forward: scaffolding is young and "seem[s]... likely to improve rapidly"; the agents were not fine-tuned and so "do not represent or bound the ARA capabilities of agents using models with additional fine-tuning"; and a roughly "100x effective compute scaleup" was untested, with task performance known to "improve continuously or abruptly depending on how it is measured." The paper's own recommendation is that ARA needs intermediate evaluation during pretraining, not just a post-hoc test of the finished model.

This sits upstream of Why do AI agents fail at workplace social interaction? — a later, more capable agent generation clearing a wider slice of office work but still bounded by the same narrow-benchmark ceiling this paper warns against. It is also an early instance of the pattern named in Does a single benchmark score actually predict agent readiness?: the paper's own limitations section is effectively a list of separate capability axes — scaffolding quality, fine-tuning, scale — that a single pilot score cannot bound. And the human-overseer role built into the evaluation, intervening on real-world side effects and adjudicating "complete" versus "partially complete," is a primitive version of the coupled unit of analysis argued for in Should we evaluate deployed agents as whole environments instead?.

The pilot measures four specific agent-scaffolding combinations on twelve hand-picked tasks in a monitored sandbox; it does not measure ARA risk for the broader population of agent designs, nor for fine-tuned or future-scale models — a limitation the paper states directly rather than leaves implicit. Its own conclusion is that the absence of ARA capability here licenses confidence only about casual, non-fine-tuned use of GPT-4- and Claude-v1.3-era models, and that fine-tuning or a 100x compute scaleup could cross the threshold without intermediate evaluation catching it first — making ARA evaluation an ongoing pretraining-time commitment rather than a one-time pass/fail gate.