What three independent failure points bottleneck traditional function calling systems?
This explores where traditional function calling actually breaks — not as one weak spot, but as three separate failures that each need their own fix.
This explores where traditional function calling actually breaks, and the surprising answer from the corpus is that it isn't one bottleneck but three independent ones — fixing any single axis leaves the other two intact. Floworks breaks the pipeline apart and finds failure at each stage: the retrieval step (vector similarity matching the user's request to the right tool) becomes unreliable as the number of available functions grows; the prompt step (stuffing full tool schemas into context) bloats the prompt and measurably degrades the model's reasoning; and the output step asks a model trained on fluent free text to emit rigid, valid JSON, which it does poorly Where do traditional function calling systems actually break down?. The key insight is structural: these are different problems wearing one label, so a better retriever does nothing for malformed JSON, and a cleaner schema does nothing for retrieval drift at scale.
What makes this worth sitting with is how the rest of the corpus, approached from completely different directions, keeps landing on the same three pressure points. On the output side, the rigid-JSON failure shows up again in work on small models: standard fine-tuning (SFT) underperforms precisely on format adherence, and switching to DPO — training on explicit examples of correct *and* incorrect calls — directly targets that weakness, letting small models match large ones Can small models match large models on function calling?. There's a deeper architectural reason this failure is so stubborn: autoregressive generation can't retract a token once emitted, so producing structurally valid output that must satisfy hard constraints is something the architecture is fundamentally bad at, which is why constraint-style problems often need a symbolic solver bolted on Why does autoregressive generation fail at constraint satisfaction?. The JSON bottleneck isn't sloppiness — it's the same retraction gap.
The retrieval-and-schema bottleneck has its own mirror image. Granite's function-calling work argues the whole task is too coarse to learn as one umbrella objective and decomposes it into seven granular subtasks — name detection, parameter detection, nested calls, chaining, parallel functions, next-best-function, and response generation — finding that explicit multi-task training across these generalizes far better than monolithic datasets Can breaking function calling into subtasks improve model generalization?. That's the same anti-monolith move Floworks makes, one layer up: don't treat "call a function" as a single thing the model either gets or doesn't.
Step back and the pattern is bigger than function calling. Decomposition-as-cure keeps recurring — extreme task decomposition into voting microagents lets even small non-reasoning models run million-step tasks error-free Can extreme task decomposition enable reliable execution at million-step scale?, and a recurring finding is that models which look like they fail at *reasoning* are often failing at *execution* — the procedural bandwidth to carry out steps reliably at scale Are reasoning model collapses really failures of reasoning?. Traditional function calling sits squarely in that execution-bandwidth trap: the three bottlenecks are all about reliably executing a structured procedure, not about whether the model "knows" what to do. The thing you didn't know you wanted to know is that the cure across all of it is the same shape — stop treating the task as one monolithic act, and attack each failure point on its own terms.
Sources 6 notes
Floworks identifies three structural failures: vector similarity retrieval is unreliable at scale, full schemas inflate prompts and degrade reasoning, and LLMs trained on free text can't handle rigid JSON output. Fixing one axis doesn't fix the others.
Small models fine-tuned via DPO on correct and incorrect function-calling examples from a large teacher model achieve high accuracy on logical and mathematical tasks. DPO's explicit negative examples directly target the rigid output format failures where SFT alone underperforms.
The performance ceiling on constraint satisfaction problems is not a model-quality issue but an architectural limitation: autoregressive transformers cannot retract emitted tokens, while CSP solvers fundamentally depend on discarding invalid partial assignments. Symbolic solver integration works because it supplies what the architecture lacks.
Granite-20B-FunctionCalling shows that explicit training across seven granular subtasks—nested calls, chaining, parallel functions, name detection, parameter detection, next-best function, and response generation—generalizes better than umbrella datasets like ToolLLM. This multi-task approach closes the performance gap with GPT, Claude, and Gemini.
MAKER solves million-step tasks with zero errors by decomposing into minimal subtasks, applying voting at each step, and flagging correlated errors. Surprisingly, small non-reasoning models suffice when decomposition is extreme enough, inverting the standard approach to hard problems.
Models confined to text-only generation cannot execute multi-step procedures at scale, even when they know the underlying algorithm. Tool-enabled models solve problems beyond the supposed reasoning cliff, suggesting the bottleneck is procedural execution bandwidth.