SYNTHESIS NOTE
Topics›Agent Harness›this note

Do skills teach procedures or inject missing facts?

This research explores whether skills help agents by providing procedural structure versus supplying new information. Understanding this distinction clarifies when and why skills improve performance.

Synthesis note · 2026-09-25 · sourced from Agent Harness

The paper asks "when do skills help, why do they work, and where do they fail," and answers that skills mostly work by anchoring procedure, not by supplying knowledge. Its abstract puts it directly: "skills work when noisy trajectories become procedural anchors that stabilize execution." Procedural anchoring "accounts for 65.7% of skill cases, versus 4.5% for explicit knowledge injection, showing that skills stabilize action rather than inject missing facts." It also reports that skills improve over Workflow Memory by 6.06 points in matched comparisons. The conclusion pairs this with the failure conditions: skills fail "when they are retrieved incorrectly, invoked in the wrong context, followed too rigidly, or used on tasks that require deeper reformulation and runtime validation."

The evidence is a contrastive design. Controlled experiments isolate "representation, outcome annotation, retrieval difficulty, and crossframework robustness," and paired trajectory analysis then explains why a skill helped or hurt in a given trial. The authors normalize 8,135 trial records, keep 238 valid unique labels from 240 open-coded records, and consolidate them into "a taxonomy of three high-level categories and twelve skill-use modes." The introduction supplies the reasoning behind the anchoring reading: repeated failures in tool-using agents "often arise not from a lack of high-level reasoning, but from rediscovering the same procedural details again and again," and a skill is "a compact description of what to do, what to check, and what pitfalls to avoid." If the missing ingredient is procedure rather than fact, a skill acts as a fixed reference that a noisy trajectory can be held against. The same property explains the failures. An anchor helps only when it is the right one, in the right context, and loosely enough held that the agent can still reformulate.

This gives a mechanism to notes that so far report gains without isolating their source. Can agents learn reusable sub-task routines from past experience? credits reusable sub-task routines for the improvement, and the anchoring finding fits that reading while adding a head-to-head comparison against "Workflow Memory," though the excerpt does not say which implementation. Can frozen language models continually improve through memory structure alone? argues that the shape of textual memory matters more than generic hints, and the 4.5% share for knowledge injection points the same way. Can a separate trained curator improve skill libraries better than frozen agents? observes that verbose generic guidance gave way to actionable structures as curation improved, which is the same distinction seen from the curator's side. The lifecycle framing also qualifies designs that treat skill generation as the bottleneck, such as How can agent systems share learned skills across users?. The paper's warning is that "building better self-evolving agents requires not only generating more skills, but also improving how agents represent, retrieve, and leverage procedural knowledge."

The excerpt is silent on several things that bear on how far the claim travels. It does not name the benchmarks, harnesses or LLMs, does not define what counts as a "skill case" in the 65.7% and 4.5% shares, lists only two of the twelve skill-use modes, and gives no success-rate effects beyond the single 6.06-point figure. It also cuts off before the three advantages skills are said to promise. What it supports is narrower and still useful: evaluate skills by what they do to trajectories, not only by aggregate success, and treat retrieval, invocation context and how strictly a skill is followed as design decisions in their own right.

Inquiring lines that read this note 12

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

How should agent systems validate and persist generated code artifacts? When should work require human-AI partnership versus full automation? What training dynamics and scale trigger emergence of reasoning capabilities? How should agents manage memory granularity to improve long-term performance? What fundamental constraints limit how effectively agents can improve themselves? What execution architectures enable agents to most effectively use tools? How do agent-learned skills transfer and improve across different tasks? How does decomposing tasks improve reasoning and prevent failure propagation?

Related concepts in this collection 4

This note in its neighbourhood — explore the map, then jump to a related concept in the list below.

Concept map
13 direct connections · 91 in 2-hop network ·medium cluster Open in graph ↗

Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph

your link semantically near linked from elsewhere

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Original note title

skills work as procedural anchors that stabilize action rather than inject missing facts — and fail when retrieved incorrectly or followed too rigidly