Saved 'how I did this before' recipes help AI agents most on unfamiliar tasks, but only if they're small and general.
When do agents benefit most from reusable workflow routines?
This explores when giving an agent a saved, reusable routine (a 'how I did this before' recipe) pays off most, and when it doesn't.
This explores when giving an agent a saved, reusable routine (a 'how I did this before' recipe) pays off most, and when it doesn't. The corpus points to a counterintuitive answer: routines help most when the new task looks least like the ones the agent has already seen, provided the routines are small, general, and used at the right moment.
The clearest evidence is Agent Workflow Memory. It saves routines at the level of sub-tasks (things like 'search for an item' or 'fill in a form') rather than whole tasks. It also strips out example-specific details and builds bigger routines from smaller ones. That gave 24.6% relative gains on Mind2Web and 51.1% on WebArena, and the gains grew as the gap between training and test tasks widened (Can agents learn reusable sub-task routines from past experience?). A whole-task recipe only helps on near-identical tasks. Small pieces can be recombined for tasks the agent has never seen.
Routines help because they steady the agent's actions, not because they hand it new facts. An analysis of 8,135 trials found that procedural anchoring explained 65.7% of skill benefits, against 4.5% for knowledge injection. The same analysis names the ways routines go wrong: retrieving the wrong one, invoking it out of context, or following it too rigidly (Do skills teach procedures or inject missing facts?). A broader framing says reliable agents move memory, skills, and protocols out of the model and into the surrounding harness. That way the model doesn't have to solve the same problem from scratch each time (Where does agent reliability actually come from?). The payoff is highest where the same sub-steps keep recurring inside otherwise varied tasks.
Routines also pay off where work is repetitive, well-defined, and needs to be checked. Most agent work is routine language tasks that small models can handle at 10–30× lower cost (Can small language models handle most agent tasks?). Production teams get more determinism from explicit, fixed function calls than from flexible tool protocols (Why do protocol-based tool integrations fail in production workflows?). Wrapping a coding agent in a skill library and persistent state produced a traceable, recoverable trail of what it did (Can orchestration layers make coding agents more auditable?). Automatically tuned harness mechanisms cut token traffic by 44.7–49.0% without losing performance, and that gain is separate from model improvements (Can agent harnesses be automatically optimized across many environments?). The same reasoning suggests routines fit best where the model's flexibility is a cost rather than a benefit.
There is a counterweight. FlowReasoner argues that fixed workflow templates are too rigid. It trains a meta-agent to design a different multi-agent workflow for each query (Can AI systems design unique multi-agent workflows per individual query?). The corpus doesn't compare the two head to head. Reading them together, a saved routine is the right tool for steps that recur across many tasks, and per-query design is the better fit when each task's overall shape is unique.
Sources 8 notes
Agent Workflow Memory induces sub-task routines at finer granularity than full tasks, abstracts example-specific values, and compounds them hierarchically. This produces 24.6% relative gain on Mind2Web and 51.1% on WebArena, with larger gains as train-test gaps widen.
Analysis of 8,135 trials shows procedural anchoring accounts for 65.7% of skill cases versus 4.5% for knowledge injection. Skills fail when retrieved incorrectly, invoked out of context, or followed too rigidly.
Research shows reliable LLM agents externalize three cognitive burdens—memory (state persistence), skills (procedural components), and protocols (structured interaction)—into a harness layer rather than relying on model scale alone. The harness unifies these externalities and eliminates the need for the model to solve the same problems repeatedly.
SLMs handle the repetitive, well-defined language tasks that constitute most agent work at 10–30× lower cost than LLMs, making heterogeneous architectures (SLMs by default, LLMs selective) the economically rational design pattern.
MCP integration caused non-deterministic failures through ambiguous tool selection and parameter inference. Replacing it with explicit direct function calls and single-tool-per-agent design restored determinism. A 306-practitioner survey confirms 85% of production teams build custom agents, forgoing frameworks.
Show all 8 sources
Dr. Claw wraps existing coding agents in persistent state objects and skill libraries, reporting higher research completeness and a traceable, recoverable process trail while keeping the underlying executor unchanged.
Cross-environment harness optimization yielded four mechanisms (action execution, context compaction, observation handling, delegated reading) that reduced token traffic by 44.7–49.0% while maintaining comparable performance on a 51-task benchmark, suggesting harness-level gains are orthogonal to model improvements.
FlowReasoner demonstrates that meta-agents trained with reinforcement learning and external execution feedback can generate unique multi-agent architectures for each user query, optimizing across performance, complexity, and efficiency—moving beyond fixed task-level workflow templates.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Demystifying Agent Skills: Why They Work-Until They Don't
- Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agents
- Real-Time Procedural Learning From Experience for AI Agents
- Towards a Science of Scaling Agent Systems
- Rethinking the Evaluation of Harness Evolution for Agents
- LatentSkill: From In-Context Textual Skills to In-Weight Latent Skills for LLM Agents
- Why Do Multi-agent LLM Systems Fail?
- RESOURCE2SKILL: Distilling Executable Agent Skills from Human-Created Multimodal Resources