Can source code replace experience as skill raw material?
Existing skill synthesis relies on agent trajectories or documents, each with limitations. Could static code repositories serve as a more reliable, scalable foundation for deriving reusable procedural skills?
The paper's claim is about where reusable skills should come from. It names two weaknesses in existing synthesis: trajectory-based skills are "inherently coupled to the environments" that produced them, and document-sourced skills "may lack evidence support and verifications." Source code, it argues, "requires no previous experience while retaining executable evidence for grounding later abstractions." Code2Skill is the pipeline built on that argument, and CodeSkillBank is its output: 1,006,822 accepted records drawn from 19,769 popular, actively maintained GitHub repositories.
The mechanism has two parts. First, selected code units are lifted into three kinds of implementation-anchored record: atomic-operation, composite-workflow, and recurring-pattern skills. Each accepted record carries workflow, boundary, provenance, and source-evidence metadata. Second, every record is verified by "source-body-blind reconstruction followed by source-aware comparison." A skill has to be specific enough that the function can be rebuilt without seeing its body, and the rebuild is then checked against the original. The evidence is a property of the record, which is why the authors call the result "auditable" as well as scalable.
The reported downstream result is that skills retrieved from CodeSkillBank lift the macro-average score from 42.90 to 47.90 across nine model settings and eight benchmarks. They win 57 of 72 protocol-matched evaluations, "especially" on coding tasks. Under a unified interface they also beat the compared trajectory-derived skill banks on all seven shared benchmarks.
This sits against the trajectory-driven skill notes in the library. Can a separate trained curator improve skill libraries better than frozen agents?, How can agent systems share learned skills across users? and Can agents learn better from their failures than successes? all treat lived experience as the raw material and put the effort into curating, aggregating or distilling it. Code2Skill removes the experience requirement: the raw material already exists as static artifacts, and its supply is bounded by the number of repositories rather than by agent runs. It also gives a different answer to the grounding problem. Where those notes lean on feedback and self-judgment, this pipeline leans on comparison against source. Can agents learn new skills without forgetting old ones? is the closest structural relative, since both hold executable skills in a retrievable library, but there the library grows through an agent's own exploration.
The excerpt leaves a lot unsaid. It does not define the baseline behind 42.90, name the eight benchmarks or the nine model settings, or say how skills are retrieved. It gives no rejection rate for the verification stage, so "accepted" carries no information about how selective it was. It does not say how a "code unit" is selected, or whether the composite and pattern records are built from atomic ones or lifted directly. The 15 of 72 evaluations without a gain are not described. The stress on coding tasks suggests domain proximity matters, but no per-domain breakdown appears. The claim that stands is narrower than "code beats experience": for the compared trajectory-derived banks, code-derived skills did better on the shared benchmarks. Whether code can supply skills for work that leaves no code behind remains open.
Inquiring lines that read this note 8
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
When should work require human-AI partnership versus full automation? How should agent systems validate and persist generated code artifacts?- How do agents retrieve and compose skills from hierarchical multimodal wikis?
- How do skill libraries from human resources compare to hand-written skill libraries?
- What makes a distilled skill verifiable and ready for agent execution?
- How does source-blind reconstruction verify that extracted skills are specific enough to be reusable?
- What metadata properties make code-derived skills auditable and comparable to their original source?
Related concepts in this collection 5
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Can a separate trained curator improve skill libraries better than frozen agents?
Explores whether decoupling skill curation from agent execution enables better long-term learning of what skills to keep, delete, or refine. Matters because manual curation doesn't scale and heuristic approaches lack feedback.
curates skills learned from task streams, while Code2Skill needs no task stream at all
-
How can agent systems share learned skills across users?
Individual users operating autonomous agents independently rediscover solutions because systems lack mechanisms to propagate discoveries. Can centralized aggregation and automatic evolution convert isolated experiences into shared capabilities?
trajectory-sourced skill supply that depends on user volume, contrasted with repository-sourced supply
-
Can agents learn better from their failures than successes?
Does storing reasoning strategies extracted from both successful and failed experiences improve agent learning compared to tracking only successes or raw trajectories? This matters because failures offer preventative lessons that successes alone cannot teach.
strategy distillation from self-judged experience, against grounding by source comparison
-
Can agents learn new skills without forgetting old ones?
Explores whether externalized skill libraries—storing learned behaviors as retrievable code rather than parameter updates—can solve the catastrophic forgetting problem that plagues continual learning systems.
executable skill library grown through exploration, where this one is mined from existing code
-
Do skills teach procedures or inject missing facts?
This research explores whether skills help agents by providing procedural structure versus supplying new information. Understanding this distinction clarifies when and why skills improve performance.
qualifies: retrieved-skill gains like Code2Skill's come mostly from procedural anchoring, not injected knowledge, and failures trace to retrieval, invocation and rigid following rather than missing content
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Grounded Skill Synthesis from Code at Scale for Agentic Intelligence
- RESOURCE2SKILL: Distilling Executable Agent Skills from Human-Created Multimodal Resources
- SkillOS: Learning Skill Curation for Self-Evolving Agents
- MUSE-Autoskill: Self-Evolving Agents via Skill Creation, Memory, Management, and Evaluation
- Repo-To-Skill: Distilling GitHub Repositories Into AI4AI Skills
- Demystifying Agent Skills: Why They Work-Until They Don't
- ColluSkill: Adversarial Cross-Skill Composition for Evading Agent Skill Scanners
- Code as Agent Harness
Original note title
skills lifted from source code need no prior experience and keep executable evidence for grounding, unlike trajectory-derived or document-sourced skills