What actually drives AI data center costs?

Epoch AI (Amelia Michael and Ben Cottier) models a "typical one-gigawatt AI data center" and finds it "requires $38 billion in up-front capital expenditure (CapEx) and $0.9 billion in annual operating expenses (OpEx)." Annualizing the CapEx over each asset's lifespan, they put "the total cost of ownership" at "$8.5 billion per year." The composition is lopsided: "Servers dominate this cost at $5 billion per year, or 60% of the total," while "operating costs are small by comparison: even energy, the largest OpEx category, costs only $0.6 billion per year." The updated figure comes to "$8.5 million/MW, down from $10.8 million/MW" in Michael's earlier 100 MW model, after scaling to 1 GW and switching the assumed hardware from NVIDIA DGX H100 to GB200 NVL72 systems.

The mechanism is a capital-recovery annualization: capital expenses (servers, facility, land, utility works, bandwidth) are multiplied by a capital recovery factor (CRF) that depends on asset lifespan and the discount rate (the computer-services industry's weighted average cost of capital), so that one-time CapEx can be compared against recurring OpEx (energy, maintenance, labor, tax, water) on the same annual basis. The authors note this annualization is "sensitive to the IT equipment lifespan": at their assumed 5-year server life the total is $8.5B/year, but "shortening the IT lifespan to 3 years raises the total annual cost to $12B," while "extending it to 7 years lowers the cost to $7B" — a swing of several billion dollars a year from a single depreciation assumption, on a model the authors describe as "stylized," not "an estimate for any specific facility."

This sits alongside Does switching between coding harnesses change model accuracy? and Do persistent agents really cost less per token? as a reminder that the inference-side cost figures those notes track ultimately rest on a capital-heavy, server-dominated infrastructure base rather than on energy or operating expense — if GPU and networking capex is 60% of a data center's annualized cost, hardware price and depreciation assumptions, not electricity prices, are the lever most likely to move the price of a token. It also offers an infrastructure-cost counterpart to How much progress have AI agents actually made on NanoGPT?'s dollar-denominated framing of AI progress: the dollars METR measures being spent on compute sit on top of a cost base this model says is mostly servers, not power.

The model does not say how this annualized cost translates into the price charged for inference, what margin or markup a cloud provider or model lab applies on top of it, or how the estimate would change for facilities built on different hardware, financing, or power-purchase terms than the "typical" US hyperscaler case assumed here. It also does not address whether the $38 billion in capex is being recovered through current revenue — only Epoch AI's own breakdown of what that capex and the associated opex consist of.