Can distilled skills close the gap in ML research agents?

The paper argues that autonomous ML research agents, built from a model backbone plus a harness for "planning, execution, memory, and verification," still leave domain-specific know-how outside the agent. It calls the gap operational knowledge, "the know-how that separates knowing a method from making it work." The payoff is stated in the discussion: under a fixed GPT-5.5 Codex setup and matched downstream budgets, the skill-equipped agent scores 134.3% higher on MLE-bench, 34.4% higher on PaperBench, 9.2% higher on FrontierCS, and 14.0% higher on PassNet. The authors read this as support for the claim that research agents "can improve by adding operational knowledge rather than relying only on stronger control loops."

The mechanism is a division of labor with a hole in the middle. The model's prior is "broad but fixed," and the harness "controls procedure but does not supply domain content." ML research is expertise-intensive, so success depends on knowing which methods and tools to use, when, and how to use them correctly, and neither component holds that. The knowledge exists in repositories and papers, but "in forms written for human readers and too large to load during a task." DisCo distills it into compact, verified skills, organized as skill graphs and loaded as operating context. It runs in two forms: task-agnostic, which condenses widely used repositories into the AREX-Skill Library (5,000+ verified skills from 1,000 repositories, organized into 20 areas and 178 capability families), and task-oriented, which produces the skills a concrete task calls for. The stated point is reuse, so the knowledge is built once rather than "rediscovered during each run."

This sits beside several notes that also treat skills as a layer separate from the executor. Can a separate trained curator improve skill libraries better than frozen agents? separates curator from executor too, but the skills there are learned from grouped task streams. DisCo's excerpt describes distilling from existing human-written artifacts. How can agent systems share learned skills across users? draws skills from usage trajectories. Here the source is repositories and papers, so no interaction history is needed. Can codified expertise let non-experts match specialist output? makes the same move of getting human domain know-how into the agent's context, from an industrial case study of a few codified rules. This paper attempts it at library scale.

The excerpt leaves several things open. It does not name the comparison condition beyond the fixed setup, and it gives no run counts, variance, or units for the percentage gains. It does not say how skills are "verified." The discussion also notes that paper-derived and task-oriented skills were constructed for the evaluated settings, so the benchmark gains may not come from the task-agnostic library alone, and the excerpt does not separate the two. What the evidence supports, at the strength stated, is that operational knowledge is a lever available alongside harness engineering. The word "only" in the authors' own claim keeps it from being a claim that skills outweigh better control loops.