What if an AI's skills were managed like code — versioned, tested, and maintained — instead of one-off prompts written and forgotten?
How should AI skills be created and managed like software artifacts?
This explores what it means to treat an agent's learned skills the way engineers treat code — authored in context, versioned, composed, curated, and compiled — rather than as one-off prompts.
This explores what it means to treat an agent's learned skills the way engineers treat code — authored in context, versioned, composed, and maintained — and the corpus converges on a surprisingly complete software lifecycle for them. Start with authoring. The old habit is to write skills offline and hand them to an agent, but that creates a mismatch between the situation the skill was written for and the situation it runs in. Creating skills *inside* the agent's own reasoning loop closes that gap: the new skill is grounded in the exact task, gets immediate feedback, and is validated at runtime before it's kept Does creating skills inside the agent loop eliminate mismatches?. Think of it as writing the function while you're in the debugger with live data, not from a spec sheet.
Once you have skills, they behave like a codebase — which means they need a package registry and a way to build bigger things from smaller ones. A skill library stored as executable, searchable entries lets an agent compose complex behaviors out of simple ones and keep learning without overwriting what it already knew, sidestepping the catastrophic forgetting that plagues approaches that just retrain weights Can agents learn new skills without forgetting old ones?. The unit of reuse can be finer than a whole task: extracting and abstracting recurring *sub-task* routines — and compounding them hierarchically — produces large gains, and the gains grow precisely as the new situations drift further from the training ones Can agents learn reusable sub-task routines from past experience?. That's the software payoff of good modularity: reuse compounds.
Here's the part you might not expect to want: a library, like any codebase, rots without maintenance, and the maintainer should be a *different* component from the worker. Separating a trainable curator from a frozen executor pushes the repository away from generic, verbose additions and toward actionable execution logic and reusable meta-strategies — and that curator generalizes across different agent backbones Can a separate trained curator improve skill libraries better than frozen agents?. This is essentially the argument for a dedicated librarian or a linter-plus-refactorer role: the thing that *does* the work is rarely the best judge of what belongs in the shared library.
Managing skills like artifacts also means you get to choose their *representation*, and there's a real build step available. Skills don't have to live as plaintext prompts eating up your context window; a hypernetwork can compile textual skills into plug-and-play LoRA adapters, cutting prompt overhead 64–72% while matching or beating in-context versions — and, because they're now weights, they compose through parameter arithmetic Can skills work better as weights than as prompts?. That's the difference between shipping interpreted source and shipping a compiled, linkable binary. And the whole enterprise rests on the same discipline that makes software maintainable: decomposing a hard task into modular, individually debuggable steps, each handed only the context it needs Can algorithms control LLM reasoning better than LLMs alone?.
The deeper move here — the thing worth walking away with — is that skill management stops being a knowledge problem and becomes a *systems* problem, complete with an evolutionary version-control story. Rather than proving a skill correct in advance, you can keep an archive of variants, benchmark them empirically, and let the ones that measurably help survive — open-ended self-improvement that reads a lot like continuous integration for an agent's own capabilities Can AI systems improve themselves through trial and error?. Authoring in-loop, composing from a library, curating with a separate trained maintainer, compiling to weights, and validating by benchmark: that's not a metaphor for software engineering, it's the actual playbook.
Sources 7 notes
MUSE-Autoskill demonstrates that invoking skill creation from within the agent's reasoning loop grounds new skills in exact task context, immediate feedback, and runtime validation. In-loop skills reach 87.94% task accuracy and transfer to other agents with minimal loss, eliminating the situated context problem of offline authoring.
VOYAGER demonstrates that storing executable skills in an embedding-indexed library and composing complex skills from simpler ones allows agents to learn continuously while avoiding the forgetting that occurs with weight-update-based methods. Environmental feedback refines skills while an automatic curriculum drives continual exploration.
Agent Workflow Memory induces sub-task routines at finer granularity than full tasks, abstracts example-specific values, and compounds them hierarchically. This produces 24.6% relative gain on Mind2Web and 51.1% on WebArena, with larger gains as train-test gaps widen.
SkillOS shows that separating a trainable curator from a frozen executor, grouped by task streams, causes skill repositories to shift from generic verbose additions toward actionable execution logic and cross-task meta-strategies. The trained curator generalizes across different executor backbones and domains.
LatentSkill uses a hypernetwork to convert textual agent skills into plug-and-play LoRA adapters, reducing prefill tokens by 64–72% while maintaining or beating in-context baselines. Weight-space skills form composable semantic structures that can be scaled and combined through parameter arithmetic.
Show all 7 sources
LLM Programs embed LLMs within explicit algorithms that manage control flow and state, presenting only step-specific context to each LLM call. This information hiding addresses capability and context window limits while treating complex reasoning as modular, debuggable sub-tasks.
DGM replaces formal proofs with empirical benchmarking and maintains an evolutionary archive of agent variants, achieving 2.5× improvement on SWE-bench and 2.2× on Polyglot by discovering capabilities like better code editing and context management.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- LatentSkill: From In-Context Textual Skills to In-Weight Latent Skills for LLM Agents
- MUSE-Autoskill: Self-Evolving Agents via Skill Creation, Memory, Management, and Evaluation
- SkillClaw: Let Skills Evolve Collectively with Agentic Evolver
- Compositional Skill Routing for LLM Agents: Decompose, Retrieve, and Compose
- SkillOS: Learning Skill Curation for Self-Evolving Agents
- Continual Learning Bench: Evaluating Frontier AI Systems in Real-World Stateful Environments
- MetaClaw: Just Talk — An Agent That Meta-Learns and Evolves in the Wild
- Agent Workflow Memory