AI agents can learn skills from real experience or from code that already exists. Which one actually teaches them more?
Can skills learned from interaction trajectories outperform skills from static repositories?
This explores whether agent skills distilled from what agents actually did (interaction trajectories) beat skills pulled from material that already exists, like GitHub code, and what the corpus says about that trade-off.
This explores whether agent skills distilled from what agents actually did beat skills pulled from material that already exists, like GitHub code. The corpus has evidence on both sides but no clean matched comparison. The one direct head-to-head favors code, and other work suggests the source may matter less than what happens to skills after they're extracted.
The case for static material comes from Code2Skill. It pulls over a million skills out of GitHub repositories and checks each one by rebuilding the source without seeing it. It raised the macro-average across eight benchmarks from 42.90 to 47.90, and it beat both trajectory-derived and document-derived alternatives (Can source code replace experience as skill raw material?). Its advantages are practical. It needs no prior agent experience, so there is no cold-start problem, and the skills stay auditable because they trace back to executable code. The gain is about five points on one paper's own comparison, so it is a real edge but a modest one.
The case for trajectories is about fit and transfer. Agent Workflow Memory extracts reusable sub-task routines from past runs and strips out example-specific values. It gained 24.6% relative on Mind2Web and 51.1% on WebArena, and the gains grew as the gap between training and test tasks widened (Can agents learn reusable sub-task routines from past experience?). That pattern suggests experience-derived routines capture how to act in an environment, which no repository documents. VOYAGER shows a related strength. Its skills are refined by environmental feedback and composed from simpler ones, so an agent can keep learning without the forgetting that weight updates cause (Can agents learn new skills without forgetting old ones?). A separate line of work finds that sequential decision-making learns best from whole trajectories from the same environment, not isolated examples (Why do trajectories matter more than individual examples for in-context learning?). Read together, these suggest trajectory skills are strongest when the trajectories come from the place you'll deploy.
The question also assumes trajectory skills and static repositories are separate camps, and the corpus blurs that. SkillOS trains a curator, separate from a frozen executor, to maintain a skill repository. Under that training the repository shifts away from generic, verbose additions and toward actionable execution logic and cross-task meta-strategies. The curator also works across different executor models and domains (Can a separate trained curator improve skill libraries better than frozen agents?). This makes curation look like the deciding factor. An untended repository fills up with noise whether its entries came from code or from experience.
My reading is that the two sources fail in different ways. Code gives breadth, verifiability and a strong start but knows nothing about your environment. Trajectories give fit and adaptation but need experience first and need filtering to avoid junk. The natural combination is to seed from code and refine from experience under a trained curator. The corpus doesn't test that combination directly, so it remains an open question.