TOPIC
Model Routers
A subject the collection covers, read through 6 synthesis notes.
View as
How quickly does behavioral diversity plateau in model pools?
When you combine multiple language models into a routing system, does adding more models keep improving behavioral coverage, or do you hit diminishing returns fast? This matters for designing efficient multi-model systems.
Multi-agent scaling follows three quantitative laws with measurable thresholds Small language models suffice for most agentic subtasks cost-effectively Routing requires behavioral diversity and stability to be meaningful Multi-agent performance is primarily determined by token spending. Multi-agent capability ceilings are structural, not scalable capability gains.
Can routing queries to task-matched structures improve RAG reasoning?
Does matching retrieval structure type to task demands—tables for analysis, graphs for inference, algorithms for planning—improve reasoning accuracy over uniform chunk retrieval? This explores whether cognitive fit principles from human learning transfer to AI systems.
When do graph databases outperform vector embeddings for retrieval? Can reasoning topologies be formally classified as graph types? Can organizing knowledge structures beat raw training data volume? Can query-time graph construction replace pre-built knowledge graphs? How should retrieval and reasoning integrate in RAG systems?
Graph databases outperform embeddings for relational domain queries requiring multi-hop traversal Reasoning topologies classify formally as path graphs, trees, and directed graphs Knowledge structure outperforms volume in domain model training Query-time logic graphs eliminate pre-built graph costs and inflexibility Retrieval and reasoning integrate most effectively through MDP formulation and process-level supervision.
What makes query complexity a better routing signal than response quality? Does sentence-level granularity capture enough structure for complex reasoning tasks? What graph structures better support multi-hop reasoning than pairwise edges? Can retrieval improve multi-step reasoning by triggering at each uncertainty? Why does visual similarity retrieval fail for embodied agents? Can task-aware ranking replace similarity scoring in other RAG systems? How should we allocate compute between reasoning and retrieval iterations? Does parallel retrieval outperform sequential search chains at test time? How can per-step decisions about knowledge retrieval improve reasoning over uniform policies? What makes proactive tool retrieval better than single-round semantic matching? What replaces truth-correspondence in probabilistic knowledge representations? How much does organized knowledge improve learning efficiency versus raw data?See all 115 inquiring lines on this note →
Can routers select the right model before generation happens?
Explores whether LLMs can be matched to queries by estimating difficulty upfront, before any generation begins. This matters because routing could cut costs significantly while preserving response quality.
Adaptive inference compute allocation outperforms fixed budgets per prompt Smaller models with more inference compute match larger ones on hard prompts Small language models suffice for most agentic subtasks cost-effectively Routing queries to specialized models outperforms frontier models Prompt effectiveness for recommendations varies sharply by model tier
Why does single-model routing beat ensemble and cascade approaches on latency? How do routing and test-time compute scaling work together as optimization axes? What makes query complexity a better routing signal than response quality? Can routing enable heterogeneous SLM-first architectures at scale? Should model routing decisions account for prompt-tier dependencies? Can this distillation pattern apply beyond e-commerce to other latency-constrained domains? Why is latency budget a constraint for e-commerce rankers? Can model routing and compute allocation work together as independent optimizations? How do byte-level models allocate compute without explicit difficulty estimators? What constraints force mobile deployments to operate in the sub-billion parameter regime? Can routing systems prevent expert models from failing outside their specialty? Does the optimal model size depend on what capabilities you actually need?See all 38 inquiring lines on this note →
What decisions must multi-agent routing systems optimize simultaneously?
Standard LLM routing only picks which model to use. But multi-agent systems involve four interdependent choices: topology, agent count, role assignment, and per-agent model selection. Does optimizing all four together actually improve performance?
Can AI systems design unique multi-agent workflows per individual query? Can multi-agent teams automatically remove their weakest members? When does adding more agents actually help systems? Can routers select the right model before generation happens? Can semantic capability vectors replace manual agent routing?
Query-level meta-agents generate personalized multi-agent systems via RL feedback Contribution scoring enables inference-time deactivation of low-performing agents Multi-agent scaling follows three quantitative laws with measurable thresholds LLM routing selects models before generation by estimating query complexity Semantic capability vectors enable scalable agent discovery without manual wiring.
Should model routing decisions account for prompt-tier dependencies? Can model routing and compute allocation work together as independent optimizations? What specific network sizes trigger coordination degradation in LLM systems? How should topology routing adapt to different task types? What structural constraints does topology impose on role and LLM assignment? Can construction-time routing and runtime agent pruning be combined effectively? How do multi-agent routers balance flexibility against interpretability in design? How do routers decide when to escalate from small to large models? Can multiple small models outperform a single large model with good routing? How does routing decide between models before generation happens? What four decisions matter most in multi-agent system routing? When and what should a model actually decide to delegate?
When does routing actually matter for multi-model systems?
Routers are judged on accuracy and cost, but what structural properties determine whether routing does anything meaningful? This explores when a router is genuinely coordinating work versus just getting lucky.
LLM routing selects models before generation by estimating query complexity Routing queries to specialized models outperforms frontier models Multi-agent routing requires optimizing four decisions at once Behavioral diversity in model pools saturates within fewer than ten agents Multi-agent scaling follows three quantitative laws with measurable thresholds
Can routing beat building one better model?
Does directing queries to specialized models via semantic clustering outperform investing in a single frontier model? This challenges whether model improvement or model selection drives performance gains.
Adaptive inference compute allocation outperforms fixed budgets per prompt Smaller models with more inference compute match larger ones on hard prompts Small language models suffice for most agentic subtasks cost-effectively LLM routing selects models before generation by estimating query complexity Routing requires behavioral diversity and stability to be meaningful
Why do naive baselines outperform trained models in entity-level CRS evaluation? Why does single-model routing beat ensemble and cascade approaches on latency? What makes query complexity a better routing signal than response quality? Can routing enable heterogeneous SLM-first architectures at scale? Should model routing decisions account for prompt-tier dependencies? Can this distillation pattern apply beyond e-commerce to other latency-constrained domains? Why is latency budget a constraint for e-commerce rankers? How does nesting optimization levels improve on traditional network depth? Can semantic clustering of stakeholders preserve meaningful evaluative diversity without manual curation? Can bilevel autoresearch discover new search mechanisms for the inner research loop? Can bilevel autoresearch succeed when the inner and outer loops use different models? Can model routing and compute allocation work together as independent optimizations?See all 68 inquiring lines on this note →