Is memory becoming the dominant cost in AI chips?

Epoch AI (Venkat Somala) estimates that high-bandwidth memory (HBM, the stacked memory chips bundled next to the processor on an AI accelerator) "has grown from 52% to 63% of total AI chip component spending between Q1 2024 and Q4 2025," averaged across chips designed by Nvidia, AMD, Google, and Amazon and weighted by production volume. Over the same period logic dies held "roughly flat near 13%," advanced packaging (the process of bonding memory and compute dies together, sometimes called CoWoS) fell from 19% to 15%, and auxiliary components fell from 15% to 9%. In dollar terms, HBM spend across the four designers "grew from roughly $12 billion in 2024 to $32 billion in 2025," which Epoch AI calls "a faster year-over-year increase than any other component," and total component spend on AI chips grew from "approximately $22 billion in 2024 to $52 billion in 2025," with HBM accounting for "roughly $20 billion of that increase." Epoch AI expects memory to "likely account for an even larger share in 2026 as memory supply remains tight and prices rise."

The estimate comes from Epoch AI's "AI Chip Components explorer," which builds a per-chip bill of materials from financial disclosures, supplier filings, and analyst reports, then multiplies per-chip cost by estimated quarterly production volume to get total spend by category. Epoch AI reports a 90% confidence interval on each component's per-chip cost and shows two uncertainty bounds on the resulting share — one where only that component sits at its 5th or 95th percentile, another where every component sits simultaneously at its most extreme value — because "both numerator and denominator are uncertain." As a sign the shift is already reaching buyers' budgets rather than staying a modeling exercise, Epoch AI notes hyperscalers pricing it into guidance: "Microsoft's $190 billion FY2026 capex outlook includes about $25 billion from higher component prices, while Meta raised its 2026 capex range by $10 billion, citing higher component prices."

This sits one layer below Will inference compute soon exceed training compute demand?, which frames chips as the base "Physical" layer of its stack; this note supplies the cost composition inside that layer and suggests that as inference demand (which that note projects will dominate compute) pushes more memory-hungry accelerator purchases, the physical layer's bill of materials is shifting toward memory rather than logic. It also gives a hardware-side counterpart to Are enterprises actually spending more on AI that works?: that note tracks spend moving into recurring IT budgets at the enterprise level, while Microsoft's and Meta's capex revisions here show the same cost pressure surfacing at the hyperscaler capex line, with memory price inflation as a specific named driver.

The excerpt is a cost-accounting estimate, not a market price series or a causal account of why memory got tight; it does not establish what is driving HBM's price rise (fab capacity, demand from a particular chip generation, supplier concentration) or whether the 2026 projection will hold. It also covers only chips from four designers, so it says nothing about memory's cost share in other accelerator families. The clear implication, qualified to that scope, is that near-term AI infrastructure cost growth is increasingly a memory-supply story rather than a logic or packaging one, which argues for watching HBM supply and pricing as a leading indicator of AI compute cost rather than treating chip cost as a single undifferentiated number.