Will AI inference demand outpace supply capacity soon?

Luke Emberson and Jaime Sevilla, writing in an Epoch AI Gradient Update, argue that global demand for LLM tokens is on a trajectory to outrun the world's inference supply, making a "compute crunch" near, "if not already here." Their demand proxies include the growth of tokens served across Google's platforms and the token intensity of heavy users such as Apple's software engineers, reportedly "permitted to use up to $300 in tokens per day," and Meta, whose "85,000 employees used 60 trillion tokens in one month." Extrapolating usage at that intensity across the roughly 30 million software engineers worldwide — with Stack Overflow's 2025 survey putting daily AI use at about 47% of developers — they estimate aggregate demand today of "between 200 million and 4 billion tokens per second, growing by roughly 10× per year."

On the supply side, they estimate the world's Blackwell GPUs could currently deliver "a combined 500 million to 20 billion output tokens per second" — enough, they note, to cover even Exponential View's estimate of total tokens processed across all providers worldwide ("around 5 billion tokens per second") using a model as capable as Kimi K2.6. But supply growth is bounded by the slower of two exponentials: compute capacity and memory bandwidth, "growing exponentially at 3.4×/year and 4.1×/year, respectively." Because compute is the long-run bottleneck, they model total inference capacity as "well modeled as growing at 3.4×/year" regardless of context length — a rate they judge "likely faster than supply is expanding" when set against roughly 10×/year demand growth. Long-context, agentic workloads such as coding are flagged as the first to feel the squeeze, since they are more memory-bound. As evidence the crunch is already underway, they point to Anthropic "reducing quotas during peak hours and incentivizing off-time usage."

This sharpens the demand side of Will inference compute soon exceed training compute demand?: where that framing argues qualitatively that inference compute will overtake training as agents proliferate, Emberson and Sevilla supply the quantitative growth-rate gap — roughly 10×/year demand against 3.4×/year supply — that would make inference capacity the binding constraint specifically for long-context, agentic use. The Meta figure also overlaps with Does token volume measure AI productivity or enable gaming?: both draw on the same 85,000-employee, trillion-token-scale usage data, though Emberson and Sevilla read it as a demand signal for crunch forecasting rather than as evidence of gamed incentives.

The authors are explicit that their proxies are "imperfect": the analysis doesn't establish the composition of tokens by model size, how much of demand growth comes from longer contexts versus more users, or how fast provider price changes are shifting usage, and it sets aside the double-edged effect of efficiency gains, which both cut the cost of a token and unlock new demand by making more use cases cost-effective. The piece does not establish that a crunch will in fact materialize, only that current trend lines point that way; its own hedge is that continued efficiency gains could let smaller, cheaper models keep pace with frontier capability, so a crunch would more likely raise the price of frontier access than cut everyday users off from AI altogether.