Buying AI chips outright versus renting them by the token — which actually costs less once you add up the whole bill?

This explores how the way AI compute is paid for (buying hardware up front and paying it off over time, versus renting capacity per token through an API) changes what AI really costs over its lifetime. The collection doesn't cover financing terms in the strict sense, such as interest rates, lease structures or loan lengths. What it does show is that the biggest cost in AI infrastructure is paying back the price of the hardware. That makes financing terms one of the biggest levers on total cost.

The clearest evidence comes from Epoch AI's cost model of a one-gigawatt data center What actually drives AI data center costs?. Of roughly $8.5B a year in total cost of ownership, servers account for about $5B. Energy, the cost people usually worry about, is only about $0.6B. When capital recovery is 60% of the bill, the assumptions behind it carry most of the weight: how fast the hardware loses value and what the money costs to borrow. A modest change in either moves the total more than a large change in electricity prices would. This is an inference from the cost structure, not something the paper tests directly, but it shows where to look.

The same logic works at the scale of a single company. A 54-scenario study of self-hosting versus paying for a commercial API When does self hosting an LLM beat commercial API costs? found that small models pay for themselves in 0.3 to 3 months. Large models take anywhere from 3.5 to almost 70 months. Self-hosting is a financing decision in disguise: you pay up front instead of per use. The longer the payback period, the more financing costs and hardware ageing eat into the savings. High-volume workloads and data-residency requirements shorten the payback, because they make the rented alternative more expensive.

Here is the twist the break-even math tends to hide: the rented alternative keeps getting cheaper. Benchmark performance per dollar improves 5 to 10 times a year Is benchmark progress outpacing the cost to achieve it?. For that reason, one paper argues that evaluations should report token counts rather than fixed dollar costs Should benchmarks report token counts instead of frozen costs?, since a cost frozen at today's prices goes stale quickly. A 69-month payback calculated against today's API prices may never actually arrive if those prices keep falling. Long-term commitments to hardware carry a risk that pay-as-you-go pricing does not. The same study adds a counterweight: the absolute cost of running frontier models is rising 3 to 18 times a year. Whether owning or renting wins therefore depends heavily on which kind of model you need.

Finally, how a system is built and used can move total cost as much as how it's paid for. In HAL's cost-controlled evaluations How much does agent design choice actually cost?, the most expensive model was rarely the best value. Scaffold choices alone (prompts, tools and control logic) changed costs by up to 9 times with little gain in accuracy. OpenRouter's 100-trillion-token analysis Does LLM price alone determine which model users choose? found that real buyers don't simply pick the cheapest option: they trade price against reliability and capability. Payment terms set the starting price, but how efficiently the system runs often decides the final bill. For actual finance modelling, such as cost-of-capital assumptions or leasing versus buying GPUs, the collection is thin, and you'd need sources outside it.