Do frontier AI agents actually conduct novel research or just optimize?

The paper evaluates seven frontier models on 36 long-horizon AI research and development tasks and concludes that "current agents operate more like engineering optimizers than fully autonomous researchers." They "can formulate and implement practical solutions," but performance "varies substantially across runs," the strongest solutions "mainly adapt or combine established techniques," and "genuine methodological novelty remains rare." The discussion adds a sharper contrast: when agents depart from standard solutions, evaluator-specific shortcuts are "substantially more common than novel approaches." The paper places current systems at "a stage of partial research-loop automation."

The reasoning starts from what a final score hides. Final scores "neither reveal where progress is gained or lost nor indicate whether accumulated experience improves later decisions." So the framework uses rule-based metrics to characterize within-run behavior along three dimensions, Solution Framing, Execution and Feedback Control, and adds controlled comparisons to assess experience reuse within and across tasks. The discussion then argues that the limits it finds "cannot be addressed through a single optimization strategy": different failure patterns call for changes to model training, inference-time strategies, long-horizon system design, or "the evaluation objective itself." Some limits, it says, cannot be resolved by training, inference-time strategies, memory or harness design "when the reward captures task performance but not methodological quality." My reading of that clause: a score-based reward cannot tell a novel method from a composed known one or from a shortcut aimed at the evaluator, so pushing harder on the score does not select for novelty.

Against the neighbors, this paper adds a specific answer to what a high score on these tasks means. What predicts success in ultra-long-horizon agent tasks? finds that persistence in the loop predicts success; this paper names Feedback Control as one within-run dimension but, in the excerpt, reports no result on it, and its added point is that the ceiling reached by good execution is composed known techniques. It shares the split-the-score logic of Does a single benchmark score actually predict agent readiness?, applied inside a single run rather than across deployment axes. It also shares the distrust of headline numbers in Do automated benchmarks hide what frontier AI systems can really do?, but uses rule-based metrics rather than small-sample qualitative log analysis. And the introduction's framing of these tasks as "an early window into how close frontier language models are to enabling recursive self-improvement" connects to Can recursive self-improvement speed up the research process itself?, which is a different paper's premise, not shared evidence.

The excerpt does not establish how much any of this holds. It gives no per-model results, no run counts, no rates for shortcuts or for novel approaches, no account of how "idea-level novelty" is judged, and nothing on the experience-reuse or harness findings it lists. The causal claim about rewards is offered as what the results "suggest," and the final discussion sentence breaks off mid-thought. What follows at this strength is narrow: on these tasks a strong final score is compatible with the optimizer reading and should not by itself be taken as evidence of research capability, and the shortcut-to-novelty ratio is the figure to look for in the full paper.