Can tutorial videos teach software agents reusable skills?
Can agents distill procedural knowledge from human-created tutorials and resources rather than learning only from their own trial-and-error? This matters because many authoring tasks require know-how that existing agent skill libraries rarely capture.
Resource2Skill argues that the raw material for agent skills does not have to be an agent's own history. The abstract says existing skill libraries are "mostly hand-written, text-centric, or derived from agent traces, leaving tutorial videos and other multimodal human resources largely underused." The framework distills tutorial videos, repositories, articles, and reference artifacts into executable skills, and the paper reports that skill access improves artifact quality by +11.9 percentage points on average over no-skill agents across seven authoring domains and four backends. It also beats two agentic-harness baselines in 26 of 28 main-aggregate model–domain cells.
The paper frames the need in terms of what authoring tasks demand: producing slide decks, spreadsheets, web pages, 3D scenes, CAD designs, and audio projects depends "less on isolated factual knowledge than on reusable procedural know-how", meaning how to decompose a goal, which tool or API pattern to use, what intermediate state to inspect, and how to recover when an operation fails. The design keeps each resource type for what it carries. Skills live in a hierarchical multimodal Skill Wiki whose entries combine structured text, code, visual examples, metadata, and provenance. Videos capture temporal operations and visual effects, code captures executable tool patterns, and articles or artifacts supply conceptual and stylistic grounding. At inference, agents retrieve and compose skills from the wiki. When coverage is insufficient, "the same construction operator can acquire new skills online", so offline construction and online gap filling share one mechanism.
Against the neighboring notes, the difference is where the skills come from. How can agent systems share learned skills across users? and Should successful and failed episodes be processed differently? both treat trajectories as the source, and Can a separate trained curator improve skill libraries better than frozen agents? curates a repository over task streams. Resource2Skill starts from artifacts people already made, which suggests a second supply line for skills that does not wait on an agent to fail and succeed its way to them. It also parallels Can codified expertise let non-experts match specialist output?, where domain knowledge is captured for the agent by hand. And Can agents learn new skills without forgetting old ones? shares the retrieve-and-compose loop.
The excerpt does not establish several things. It gives no definition of the overall score, no sample sizes, and no statement of what the two harness baselines are. It does not compare against hand-written or trace-derived skill libraries directly, so the claim that human resources are a better source is not tested in what we can see. It does not report how well online acquisition works, what construction costs, or how much resource quality matters. The safe reading is narrower: for these authoring domains, skills distilled from human-created resources beat both no skills and the harness baselines. Whether they beat other skill sources is left open.
Inquiring lines that read this note 3
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
When should work require human-AI partnership versus full automation? How should agent systems validate and persist generated code artifacts?Related concepts in this collection 6
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
How can agent systems share learned skills across users?
Individual users operating autonomous agents independently rediscover solutions because systems lack mechanisms to propagate discoveries. Can centralized aggregation and automatic evolution convert isolated experiences into shared capabilities?
a trajectory-sourced skill pipeline; Resource2Skill draws on human-made resources instead of aggregated agent traces.
-
Should successful and failed episodes be processed differently?
Explores whether asymmetric treatment of trajectories—preserving successes as full demonstrations while abstracting failures into lessons—could improve both the utility and efficiency of memory in reinforcement learning agents.
another trace-derived skill library, contrasting with distillation from tutorials, repositories, and articles.
-
Can a separate trained curator improve skill libraries better than frozen agents?
Explores whether decoupling skill curation from agent execution enables better long-term learning of what skills to keep, delete, or refine. Matters because manual curation doesn't scale and heuristic approaches lack feedback.
curates skill repositories over task streams; Resource2Skill instead constructs its wiki from external resources.
-
Can codified expertise let non-experts match specialist output?
When domain knowledge is captured as explicit rules and principles in an AI agent's scaffolding, can non-experts produce work at expert quality levels without consuming scarce specialist time? This explores whether structured knowledge codification dissolves organizational bottlenecks.
human expertise reaches the agent through hand-codified rules; here it reaches it through existing resources.
-
Can agents learn new skills without forgetting old ones?
Explores whether externalized skill libraries—storing learned behaviors as retrievable code rather than parameter updates—can solve the catastrophic forgetting problem that plagues continual learning systems.
shares the retrieve-and-compose use of an executable skill library.
-
Do skills teach procedures or inject missing facts?
This research explores whether skills help agents by providing procedural structure versus supplying new information. Understanding this distinction clarifies when and why skills improve performance.
Qualifies: skills act mainly as procedural anchors rather than injected knowledge, and fail through faulty retrieval, invocation or rigid following, which bounds A's gains
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- RESOURCE2SKILL: Distilling Executable Agent Skills from Human-Created Multimodal Resources
- ColluSkill: Adversarial Cross-Skill Composition for Evading Agent Skill Scanners
- Demystifying Agent Skills: Why They Work-Until They Don't
- SkillClaw: Let Skills Evolve Collectively with Agentic Evolver
- MUSE-Autoskill: Self-Evolving Agents via Skill Creation, Memory, Management, and Evaluation
- LatentSkill: From In-Context Textual Skills to In-Weight Latent Skills for LLM Agents
- Compositional Skill Routing for LLM Agents: Decompose, Retrieve, and Compose
- Grounded Skill Synthesis from Code at Scale for Agentic Intelligence
Original note title
distilling skills from human-created multimodal resources gives software agents reusable procedural knowledge that improves on no-skill agents