Do skills teach procedures or inject missing facts?
This research explores whether skills help agents by providing procedural structure versus supplying new information. Understanding this distinction clarifies when and why skills improve performance.
The paper asks "when do skills help, why do they work, and where do they fail," and answers that skills mostly work by anchoring procedure, not by supplying knowledge. Its abstract puts it directly: "skills work when noisy trajectories become procedural anchors that stabilize execution." Procedural anchoring "accounts for 65.7% of skill cases, versus 4.5% for explicit knowledge injection, showing that skills stabilize action rather than inject missing facts." It also reports that skills improve over Workflow Memory by 6.06 points in matched comparisons. The conclusion pairs this with the failure conditions: skills fail "when they are retrieved incorrectly, invoked in the wrong context, followed too rigidly, or used on tasks that require deeper reformulation and runtime validation."
The evidence is a contrastive design. Controlled experiments isolate "representation, outcome annotation, retrieval difficulty, and crossframework robustness," and paired trajectory analysis then explains why a skill helped or hurt in a given trial. The authors normalize 8,135 trial records, keep 238 valid unique labels from 240 open-coded records, and consolidate them into "a taxonomy of three high-level categories and twelve skill-use modes." The introduction supplies the reasoning behind the anchoring reading: repeated failures in tool-using agents "often arise not from a lack of high-level reasoning, but from rediscovering the same procedural details again and again," and a skill is "a compact description of what to do, what to check, and what pitfalls to avoid." If the missing ingredient is procedure rather than fact, a skill acts as a fixed reference that a noisy trajectory can be held against. The same property explains the failures. An anchor helps only when it is the right one, in the right context, and loosely enough held that the agent can still reformulate.
This gives a mechanism to notes that so far report gains without isolating their source. Can agents learn reusable sub-task routines from past experience? credits reusable sub-task routines for the improvement, and the anchoring finding fits that reading while adding a head-to-head comparison against "Workflow Memory," though the excerpt does not say which implementation. Can frozen language models continually improve through memory structure alone? argues that the shape of textual memory matters more than generic hints, and the 4.5% share for knowledge injection points the same way. Can a separate trained curator improve skill libraries better than frozen agents? observes that verbose generic guidance gave way to actionable structures as curation improved, which is the same distinction seen from the curator's side. The lifecycle framing also qualifies designs that treat skill generation as the bottleneck, such as How can agent systems share learned skills across users?. The paper's warning is that "building better self-evolving agents requires not only generating more skills, but also improving how agents represent, retrieve, and leverage procedural knowledge."
The excerpt is silent on several things that bear on how far the claim travels. It does not name the benchmarks, harnesses or LLMs, does not define what counts as a "skill case" in the 65.7% and 4.5% shares, lists only two of the twelve skill-use modes, and gives no success-rate effects beyond the single 6.06-point figure. It also cuts off before the three advantages skills are said to promise. What it supports is narrower and still useful: evaluate skills by what they do to trajectories, not only by aggregate success, and treat retrieval, invocation context and how strictly a skill is followed as design decisions in their own right.
Inquiring lines that read this note 12
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
How should agent systems validate and persist generated code artifacts?- Can agents acquire new skills online when offline skill coverage runs out?
- How do agents retrieve and compose skills from hierarchical multimodal wikis?
- How do skill libraries from human resources compare to hand-written skill libraries?
- Why treat tutorial videos as a separate supply line from agent trajectories?
- What makes a distilled skill verifiable and ready for agent execution?
Related concepts in this collection 4
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Can agents learn reusable sub-task routines from past experience?
Do web agents fail at long-horizon tasks because they cannot extract and reuse workflows shared across similar problems? This explores whether sub-task abstraction enables skill accumulation rather than task-by-task problem solving.
AWM reports routine reuse gains; this paper attributes skill gains to procedural anchoring and compares skills against Workflow Memory directly
-
Can frozen language models continually improve through memory structure alone?
If agents can't update parameters, what form of textual memory lets them keep learning across trials and transfer to new tasks without retraining?
both suggest the form of stored guidance matters more than its informational content
-
Can a separate trained curator improve skill libraries better than frozen agents?
Explores whether decoupling skill curation from agent execution enables better long-term learning of what skills to keep, delete, or refine. Matters because manual curation doesn't scale and heuristic approaches lack feedback.
SkillOS sees generic guidance give way to actionable structure; this paper explains why with anchoring
-
How can agent systems share learned skills across users?
Individual users operating autonomous agents independently rediscover solutions because systems lack mechanisms to propagate discoveries. Can centralized aggregation and automatic evolution convert isolated experiences into shared capabilities?
qualifies skill propagation designs, since more skills without better retrieval and use does not suffice
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Demystifying Agent Skills: Why They Work-Until They Don't
- LatentSkill: From In-Context Textual Skills to In-Weight Latent Skills for LLM Agents
- RESOURCE2SKILL: Distilling Executable Agent Skills from Human-Created Multimodal Resources
- How AI Impacts Skill Formation
- SkillClaw: Let Skills Evolve Collectively with Agentic Evolver
- SkillOS: Learning Skill Curation for Self-Evolving Agents
- Real-Time Procedural Learning From Experience for AI Agents
- SkillOpt: Executive Strategy for Self-Evolving Agent Skills
Original note title
skills work as procedural anchors that stabilize action rather than inject missing facts — and fail when retrieved incorrectly or followed too rigidly