Does dynamic tool discovery break prompt caching benefits?

Beyond its headline cost numbers, "Don't Break the Cache" identifies a second, architectural cause of lost cache benefit: "dynamic function calling can break the cache when tool definitions change between requests." It names the Model Context Protocol (MCP) specifically — "modern agentic systems increasingly leverage dynamic tool discovery through protocols such as [MCP], where available tools may vary based on connected servers or runtime context" — and states the fix as "a practical strategy is to maintain a fixed set of general-purpose, reusable functions while implementing dynamic capabilities through code generation rather than traditional function calling."

The mechanism follows directly from how prompt caching works: a cache hit requires the new request's prefix to match a previously cached prefix token-for-token. When tool definitions are serialized into that prefix and the available tool set shifts — because an MCP server was added, dropped, or returned a different manifest — the prefix no longer matches, and the cache breaks for that request. The same logic indicts common context-hygiene practices: "techniques such as summarizing or pruning old tool calls... break cached representations, making tool call caching counterproductive." Context management and cache preservation pull in opposite directions once a session is long enough to want both.

This gives Why do protocol-based tool integrations fail in production workflows? a second, independent reason to avoid MCP in production: that note's case was reliability (MCP's protocol-metadata reasoning produced ambiguous tool selection and non-reproducible failures), argued from a podcast-generation workflow with no caching angle. This paper's case is cost (MCP's variable tool manifests invalidate the cached prefix), argued from controlled caching measurements with no reliability angle. The two papers reach the same prescription — fixed, explicit functions over protocol-mediated discovery — from mechanisms that do not overlap, which is a stronger basis for the prescription than either paper alone. It is the direct companion to Does prompt caching actually reduce agentic LLM costs?, which covers this paper's cost/latency headline.

The excerpt does not isolate how much of the 41-80% cost range or 13-31% TTFT range is attributable to dynamic tool discovery specifically, as opposed to other cache-breaking content (timestamps, session identifiers, pruned history) — the fixed-functions-plus-code-generation recommendation is presented as a design heuristic that follows from the caching mechanism, not as a separately ablated result. It likewise does not report having tested code generation as a dynamic-capability substitute; it is proposed, not measured, in this excerpt. The implication holds at the strength of a mechanism-based argument: any practice that varies the tool set or prunes tool-call history across requests trades cache yield for that practice's own benefit, and the trade should be made deliberately rather than by default.