When AI task routing becomes a pipeline, one weak link early silently limits everything the system can accomplish.
What cascading bottlenecks appear when skill routing is decomposed into stages?
This explores what happens when you break skill routing into a pipeline — decompose the task, retrieve skills, compose them — and how failure at one stage silently caps the stages after it.
This explores what happens when skill routing stops being "pick the right tool" and becomes a multi-stage pipeline: real agent tasks need several skills stitched together, so routing turns into decompose-retrieve-compose — break the query into atomic sub-tasks, fetch a skill for each, and assemble them into a dependency-aware plan How should agents route across thousands of skills?. The moment you stage it that way, the stages stop being independent. Each one inherits the quality of the one before it, and the earliest weak link quietly caps everything downstream.
The first bottleneck sits at the very front: how finely you cut the task. Standard LLM decomposition lands only about 34% step-level recall, and because retrieval runs per sub-task, a missed or mis-sized step is a skill that never gets fetched no matter how good your retriever is What blocks skill retrieval in task decomposition?. The striking part is what happens when you fix it — correcting the step count recovers roughly 75% of the achievable gains, which moves the bottleneck rather than removing it. Now the limiting factor is representation-level reranking (does the retriever rank the *right* skill first?), not the earlier worry of vocabulary alignment. That's the cascade in miniature: solve granularity and the pressure migrates one stage down.
There's a deeper reason staging helps at all, which the corpus frames as a division of labor. Splitting the planner from the executor beats a single monolithic model, and — importantly — the two halves generalize differently: decomposition ability transfers across domains while solving ability does not Does separating planning from execution improve reasoning accuracy?. So the front of the pipeline is both the most fragile stage *and* the most reusable one, which is exactly why its errors are so expensive. RL training tells the same story from the inside: models first master execution correctness, then hit a wall where strategic planning becomes the binding constraint, with planning tokens carrying the entropy and the gains Does RL training follow a predictable two-phase learning sequence?. Bottlenecks don't stay put — they walk from execution toward planning as the easy stage gets solved.
The far end of the pipeline has its own quiet failure: the skill library itself degrades. If a frozen agent keeps appending skills, the repository drifts toward generic, verbose entries; a *separately trained* curator instead evolves it toward actionable execution logic and cross-task meta-strategies Can a separate trained curator improve skill libraries better than frozen agents?. So retrieval quality isn't fixed — it's a function of who's been maintaining the shelf. And at scale, the retrieval substrate matters too: coupling semantic matching with policy and budget constraints in a searchable index keeps discovery from becoming its own choke point as the number of skills and agents grows Can semantic capability vectors replace manual agent routing?.
The payoff worth taking away: staging skill routing doesn't eliminate the hard problem, it *relocates* it — from vocabulary to granularity, from granularity to reranking, from execution to planning, from retrieval to library upkeep. Each fix un-gates the next constraint. This is why routing-as-selection keeps outperforming just building a bigger model — sending queries to the right specialist per semantic cluster can beat a frontier model outright, or match it far cheaper Can routing beat building one better model? — but only if every upstream stage feeds it a clean signal. The whole point of decomposing routing is to make each bottleneck visible and separately fixable; the risk is that if you don't watch the handoffs, the weakest stage sets the ceiling for all of them.
Sources 7 notes
Real agent tasks demand composing multiple skills rather than selecting one. The routing problem becomes decompose-retrieve-compose: break queries into atomic sub-tasks, retrieve skills per sub-task, and assemble them into dependency-aware plans.
Standard LLM decomposition reaches only 34% step-level recall, gating retrieval success. Correcting step count recovers 75% of gains in iterative methods, shifting the bottleneck to representation-level reranking rather than vocabulary alignment.
Modular architectures with separate decomposer and solver models outperform monolithic LLMs, with decomposition ability transferring across domains while solving ability does not. The separation prevents planning-execution interference and produces more generalizable skills.
Across eight models, RL training consistently shows a first phase where execution correctness drives learning, followed by a second phase where strategic planning becomes the bottleneck. Planning token entropy increases while execution entropy stabilizes, and concentration of optimization on planning tokens yields significant performance gains.
SkillOS shows that separating a trainable curator from a frozen executor, grouped by task streams, causes skill repositories to shift from generic verbose additions toward actionable execution logic and cross-task meta-strategies. The trained curator generalizes across different executor backbones and domains.
Show all 7 sources
Versioned Capability Vectors embedded in HNSW indices couple semantic matching with policy and budget constraints, making capability discovery a first-class operation that scales sub-linearly as agent heterogeneity increases.
Avengers-Pro achieves 7% higher accuracy than GPT-5-medium by routing queries to optimal models per semantic cluster, or matches its performance at 27% lower cost. Ten 7B models with routing previously surpassed GPT-4.1 and 4.5, suggesting selection is a stronger lever than scaling.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Compositional Skill Routing for LLM Agents: Decompose, Retrieve, and Compose
- Divide-or-Conquer? Which Part Should You Distill Your LLM?
- MUSE-Autoskill: Self-Evolving Agents via Skill Creation, Memory, Management, and Evaluation
- MasRouter: Learning to Route LLMs for Multi-Agent Systems
- Algorithm of Thoughts: Enhancing Exploration of Ideas in Large Language Models
- When is Routing Meaningful? Diversity and Robustness in Language Model Societies
- LatentSkill: From In-Context Textual Skills to In-Weight Latent Skills for LLM Agents
- Distilling LLMs' Decomposition Abilities into Compact Language Models