An AI agent's tool-picking fails not because its index is bad, but because it carves the task into the wrong-sized steps.
Why does task decomposition granularity become the bottleneck in skill routing?
This explores why the *grain size* at which a task gets broken into sub-steps — not the search index or the skill vocabulary — is what limits an agent's ability to route work to the right skills.
This explores why the *grain size* at which a task gets broken into sub-steps ends up gating everything downstream in skill routing. The core claim in the corpus is blunt: when an agent has to pick from a large library of skills, the hard part isn't matching a sub-task to a skill — it's carving the task into the right sub-tasks in the first place. Standard LLM decomposition only recovers about 34% of the correct steps, and that step-level miss rate caps how well retrieval can ever do, no matter how good the index is What blocks skill retrieval in task decomposition?. Tellingly, once you correct the step *count*, most of the gains come back — which relocates the remaining bottleneck to reranking representations rather than to vocabulary alignment. If you split too coarsely, one 'step' spans several skills and no single skill matches; too finely, and you fragment intent so no skill matches either.
The reason this matters so much is that real agent work is composition, not selection. Routing across thousands of skills is really a decompose → retrieve → compose loop: break the query into atomic sub-tasks, fetch a skill per sub-task, then assemble them into a dependency-aware plan How should agents route across thousands of skills?. Every stage after decomposition inherits its errors. Retrieval can only find skills for the sub-tasks you actually named, and the plan can only be as coherent as the boundaries you drew. So decomposition isn't one step among many — it's the step that defines the search space for all the others.
What's interesting is that the corpus suggests granularity is a *learnable, transferable* skill rather than a fixed property of the model. When you split an agent into a separate decomposer and a solver, the decomposition ability generalizes across domains while solving ability does not Does separating planning from execution improve reasoning accuracy? — evidence that knowing *how finely to cut* is a distinct competence worth training on its own. Agents can even mine reusable sub-task routines from past runs at a finer granularity than whole tasks, compounding them hierarchically for 24–51% gains Can agents learn reusable sub-task routines from past experience?. And a trained skill curator, decoupled from a frozen executor, learns to reshape a skill library toward the granularity that actually gets reused Can a separate trained curator improve skill libraries better than frozen agents?.
The counterintuitive payoff shows up at the extreme end. Push decomposition far enough — down to minimal, near-trivial subtasks with a vote at each step — and you can run million-step tasks with zero errors, and small non-reasoning models suddenly suffice where you'd expect to need a frontier model Can extreme task decomposition enable reliable execution at million-step scale?. That inverts the usual intuition: the granularity of the cut, not the raw power of the router or executor, is the lever. The same logic appears in reasoning structured as recursive subtask trees, where the right decomposition lets a single model absorb work that otherwise needs a multi-agent system Can recursive subtask trees overcome context window limits?, and in LLM-programs that hand each step only its relevant context so sub-tasks stay modular and debuggable Can algorithms control LLM reasoning better than LLMs alone?.
So the bottleneck is granularity because it sits *upstream* of matching. You can build capability vectors and sub-linear indices that make discovery scale beautifully Can semantic capability vectors replace manual agent routing?, but a perfect index over the wrong sub-tasks still routes to the wrong skills. Get the grain right and cheap models plus simple retrieval win; get it wrong and no amount of routing sophistication recovers.
Sources 9 notes
Standard LLM decomposition reaches only 34% step-level recall, gating retrieval success. Correcting step count recovers 75% of gains in iterative methods, shifting the bottleneck to representation-level reranking rather than vocabulary alignment.
Real agent tasks demand composing multiple skills rather than selecting one. The routing problem becomes decompose-retrieve-compose: break queries into atomic sub-tasks, retrieve skills per sub-task, and assemble them into dependency-aware plans.
Modular architectures with separate decomposer and solver models outperform monolithic LLMs, with decomposition ability transferring across domains while solving ability does not. The separation prevents planning-execution interference and produces more generalizable skills.
Agent Workflow Memory induces sub-task routines at finer granularity than full tasks, abstracts example-specific values, and compounds them hierarchically. This produces 24.6% relative gain on Mind2Web and 51.1% on WebArena, with larger gains as train-test gaps widen.
SkillOS shows that separating a trainable curator from a frozen executor, grouped by task streams, causes skill repositories to shift from generic verbose additions toward actionable execution logic and cross-task meta-strategies. The trained curator generalizes across different executor backbones and domains.
Show all 9 sources
MAKER solves million-step tasks with zero errors by decomposing into minimal subtasks, applying voting at each step, and flagging correlated errors. Surprisingly, small non-reasoning models suffice when decomposition is extreme enough, inverting the standard approach to hard problems.
The Thread Inference Model demonstrates that reasoning structured as recursive subtask trees with rule-based KV cache pruning sustains accurate reasoning beyond context limits, even when manipulating 90% of the cache. This enables single models to replace multi-agent systems by handling full recursive reasoning internally.
LLM Programs embed LLMs within explicit algorithms that manage control flow and state, presenting only step-specific context to each LLM call. This information hiding addresses capability and context window limits while treating complex reasoning as modular, debuggable sub-tasks.
Versioned Capability Vectors embedded in HNSW indices couple semantic matching with policy and budget constraints, making capability discovery a first-class operation that scales sub-linearly as agent heterogeneity increases.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Compositional Skill Routing for LLM Agents: Decompose, Retrieve, and Compose
- MUSE-Autoskill: Self-Evolving Agents via Skill Creation, Memory, Management, and Evaluation
- Divide-or-Conquer? Which Part Should You Distill Your LLM?
- LatentSkill: From In-Context Textual Skills to In-Weight Latent Skills for LLM Agents
- Algorithm of Thoughts: Enhancing Exploration of Ideas in Large Language Models
- Agent Workflow Memory
- SkillClaw: Let Skills Evolve Collectively with Agentic Evolver
- How Many Instructions Can LLMs Follow at Once?