Give an AI agent a neat summary of its past lessons and it mostly shrugs, yet it reacts strongly to raw history.
Why do agents systematically ignore condensed experience in their skill documents?
This explores why an agent that is handed a distilled write-up of past lessons (a skill document) barely changes its behavior because of it, even though it responds strongly to its raw history.
This explores why an agent that is handed a distilled write-up of past lessons (a skill document) barely changes its behavior because of it, even though it responds strongly to its raw history. The corpus has a direct experiment on this. Across 10 models and 9 environments, perturbing an agent's raw experience changed what it did significantly. Editing the condensed summaries had minimal effect Why do LLM agents ignore condensed experience summaries?. The agents aren't broken. They are following the source that still contains usable information.
The same paper names three causes. First, summaries lose critical details, and the specifics that made a lesson actionable are the first things compression cuts. Second, models favor what is immediately in their context over what was retrieved and prepended. Third, pretrained knowledge already gives the model a confident way to do the task, so it has less reason to lean on external experience. A skill document is compressed, competes with a live context, and often tells the model something it already believes. That combination is easy to ignore.
This undercuts a popular pitch for skill libraries: externalize procedures into a harness so the model never has to solve the same problem twice Where does agent reliability actually come from?, Can agents learn new skills without forgetting old ones?. That pitch assumes the model actually reads and obeys what is stored. Other notes in the corpus point to fixes, though none tests them against this specific ignoring problem. On curation, a trained curator kept separate from a frozen executor pushes skill repositories away from generic, verbose additions. It moves them toward actionable execution logic and cross-task meta-strategies Can a separate trained curator improve skill libraries better than frozen agents?. That is content a model has something to act on. On structure, autonomous memory folding into episodic, working, and tool schemas avoids the degradation that badly designed consolidation causes Can agents compress their own memory without losing critical details?.
A second family of fixes changes how the experience reaches the model. Letting the agent reconstruct memory by traversing it while it reasons beats retrieve-then-reason by up to 23% Can agents reconstruct memory on demand instead of retrieving it?, because the agent pulls what it needs instead of being handed a summary. Training models to delegate subtasks and integrate the results beats passive compression, and that skill transfers to single-agent work Can delegation teach models to manage context more actively?. The most radical option skips the context window: compiling text skills into LoRA adapters cuts prefill tokens by 64–72% while matching or beating in-context baselines Can skills work better as weights than as prompts?. If part of the problem is skills losing a fight with immediate context, weights remove the fight. That is an inference from these notes, not something they test.
The practical takeaway is the experiment itself. Perturb the skill document and check whether behavior moves. A skill the agent can ignore without any change in behavior isn't carrying information the agent didn't already have.
Sources 8 notes
Across 10 LLM models and 9 environments, perturbing raw experience changed agent behavior significantly, while altering condensed experience had minimal effect. Three causes drive this asymmetry: summaries lose critical details, models favor immediate context over retrieved information, and pretrained knowledge reduces reliance on external experience.
Research shows reliable LLM agents externalize three cognitive burdens—memory (state persistence), skills (procedural components), and protocols (structured interaction)—into a harness layer rather than relying on model scale alone. The harness unifies these externalities and eliminates the need for the model to solve the same problems repeatedly.
VOYAGER demonstrates that storing executable skills in an embedding-indexed library and composing complex skills from simpler ones allows agents to learn continuously while avoiding the forgetting that occurs with weight-update-based methods. Environmental feedback refines skills while an automatic curriculum drives continual exploration.
SkillOS shows that separating a trainable curator from a frozen executor, grouped by task streams, causes skill repositories to shift from generic verbose additions toward actionable execution logic and cross-task meta-strategies. The trained curator generalizes across different executor backbones and domains.
DeepAgent's autonomous memory folding consolidates interaction history into episodic, working, and tool memory schemas. This reduces token overhead while letting agents pause to reconsider strategies—the autonomy and structure together avoid degradation that plagues poorly designed consolidation.
Show all 8 sources
MRAgent achieves up to 23% gains on reasoning tasks by reconstructing memory through active graph traversal that prunes paths based on accumulated evidence, while reducing token and runtime cost compared to fixed-retrieval pipelines.
SearchSwarm shows that training models to delegate subtasks and integrate summarized results beats passive compression, with a 30B model matching much larger ones. Critically, the delegation skill transfers to single-agent tasks, suggesting it teaches disciplined decomposition and evidence grounding, not just orchestration.
LatentSkill uses a hypernetwork to convert textual agent skills into plug-and-play LoRA adapters, reducing prefill tokens by 64–72% while maintaining or beating in-context baselines. Weight-space skills form composable semantic structures that can be scaled and combined through parameter arithmetic.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Demystifying Agent Skills: Why They Work-Until They Don't
- LatentSkill: From In-Context Textual Skills to In-Weight Latent Skills for LLM Agents
- MUSE-Autoskill: Self-Evolving Agents via Skill Creation, Memory, Management, and Evaluation
- Useful Memories Become Faulty When Continuously Updated by LLMs
- SkillOS: Learning Skill Curation for Self-Evolving Agents
- SkillClaw: Let Skills Evolve Collectively with Agentic Evolver
- Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineering
- Know It, Act on It: Investigating Memory Utilization in LLM Personalization