Can an AI learn its skills by studying existing code and expertise, instead of waiting for someone to write them by hand?
How do skill libraries from human resources compare to hand-written skill libraries?
This explores how skill libraries built by mining material humans already produced (source code, a colleague's expertise) compare to skill libraries a person writes by hand, reading "human resources" as human-made source material rather than the HR department.
This explores how skill libraries built by mining material humans already produced (source code, a colleague's expertise) compare to skill libraries a person writes by hand. The corpus has no head-to-head test of mined versus hand-written skills. It does show what mining can do, what happens to a skill library once it exists, and where the real bottleneck sits.
The strongest case for mining is code. Code2Skill pulled over a million auditable skills out of GitHub repositories, with no prior agent experience needed, and each skill was checked by rebuilding the source without seeing it. Adding those skills lifted the macro-average across eight benchmarks from 42.90 to 47.90, and they beat skills derived from agent trajectories or from documents (Can source code replace experience as skill raw material?). The corpus's comparison is against other automated sources, not against hand-writing. Its point is that mined skills keep executable evidence, so you can check them rather than trust them.
The other human source is a person. COLLEAGUE.SKILL distills someone's expertise into versioned files that can be inspected, corrected and rolled back, instead of living as hidden prompt state. It keeps what the person knows on a separate track from how they behave, so each can be audited on its own (Can person-grounded skills remain auditable without hidden prompt state?). This narrows the gap with hand-writing. A distilled skill is treated like a hand-written file, and it needs the same governance.
A library, however it began, doesn't have to stay as first written. SkillOpt trains a skill document like model weights, using a text-space optimizer that accepts an edit only if it improves held-out performance. It matches or beats baselines, adds no inference cost, and transfers across models (Can skill documents be optimized like neural network weights?). SkillOS separates a trained curator from the agent that executes skills. Under that curator, a repository moves away from generic, verbose additions toward actionable execution logic and cross-task meta-strategies (Can a separate trained curator improve skill libraries better than frozen agents?). So where a skill came from matters less than whether something keeps testing and pruning it.
Two findings suggest why scale changes the problem. Across 8,135 trials, skills mostly work by anchoring procedure (65.7% of cases) and rarely by injecting missing facts (4.5%). They fail when retrieved wrongly, invoked out of context, or followed too rigidly (Do skills teach procedures or inject missing facts?). A million mined skills make that retrieval failure the main risk. Real tasks need several skills composed together (How should agents route across thousands of skills?), and standard decomposition reaches only 34% step-level recall, which caps how well retrieval can work (What blocks skill retrieval in task decomposition?). Once skills can be mined at scale, writing them is no longer the hard part. Finding and combining the right ones is.
Sources 7 notes
Code2Skill extracts over 1 million auditable skills from GitHub repositories through source-blind reconstruction verification, improving performance from 42.90 to 47.90 macro-average across eight benchmarks, outperforming trajectory and document-derived alternatives.
COLLEAGUE.SKILL treats distilled expertise as versioned files subject to inspection, correction, and rollback—not hidden prompt state. Separating capability tracks from behavior tracks enables independent audit of what someone knows versus how they act.
SkillOpt treats skill documents as trainable external state of frozen agents, using a text-space optimizer with held-out validation gating to accept only edits that improve performance. Across 52 benchmark cells and seven models, the approach matches or exceeds baselines while adding zero inference cost and enabling transfer across models.
SkillOS shows that separating a trainable curator from a frozen executor, grouped by task streams, causes skill repositories to shift from generic verbose additions toward actionable execution logic and cross-task meta-strategies. The trained curator generalizes across different executor backbones and domains.
Analysis of 8,135 trials shows procedural anchoring accounts for 65.7% of skill cases versus 4.5% for knowledge injection. Skills fail when retrieved incorrectly, invoked out of context, or followed too rigidly.
Show all 7 sources
Real agent tasks demand composing multiple skills rather than selecting one. The routing problem becomes decompose-retrieve-compose: break queries into atomic sub-tasks, retrieve skills per sub-task, and assemble them into dependency-aware plans.
Standard LLM decomposition reaches only 34% step-level recall, gating retrieval success. Correcting step count recovers 75% of gains in iterative methods, shifting the bottleneck to representation-level reranking rather than vocabulary alignment.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Demystifying Agent Skills: Why They Work-Until They Don't
- Compositional Skill Routing for LLM Agents: Decompose, Retrieve, and Compose
- SkillOS: Learning Skill Curation for Self-Evolving Agents
- RESOURCE2SKILL: Distilling Executable Agent Skills from Human-Created Multimodal Resources
- MUSE-Autoskill: Self-Evolving Agents via Skill Creation, Memory, Management, and Evaluation
- Grounded Skill Synthesis from Code at Scale for Agentic Intelligence
- LatentSkill: From In-Context Textual Skills to In-Weight Latent Skills for LLM Agents
- SkillOpt: Executive Strategy for Self-Evolving Agent Skills