If AI models keep getting cheaper and faster, does that satisfy demand for them — or just create more demand?
Can inference demand itself change based on model price and speed?
This explores whether the amount of AI computation people and systems use goes up or down as models get cheaper and faster, rather than staying a fixed quantity that the industry only has to supply.
This explores whether inference demand is a fixed amount the industry has to meet, or whether it grows and shrinks with price and speed. The corpus has no direct study of price elasticity. Read together, though, its notes point to a clear pattern: cheaper inference does not seem to lower total demand. And a surprising share of that demand is set by how the models are built, not by how many people use them.
Start with two notes that look like they contradict each other. Epoch AI's Denain finds that the cost of reaching a given capability level falls roughly 5–10x a year, through distillation, better algorithms and better hardware Does inference cost remain a persistent economic burden?. Emberson and Sevilla estimate that token demand grows about 10x a year while inference supply grows only about 3.4x. They see a coming compute crunch, with early signs such as Anthropic cutting usage quotas during peak hours Will AI inference demand outpace supply capacity soon?. If costs are collapsing, why would there be a shortage? The likely answer is that cheaper tokens get used for more things, so total demand rises faster than costs fall. That is a classic rebound effect. The corpus implies it but doesn't measure it.
The less obvious point is that a lot of 'demand' is not people sending more queries. It is models spending more tokens on each query. Research on test-time compute shows that a smaller model given more thinking time can match a much larger one on hard problems Can inference compute replace scaling up model size?. So a cheap model doesn't necessarily use less compute. It may just move the spending from training to inference. Reasoning models make this a lasting trade, because their training makes extra tokens useful in a way that non-reasoning models can't match however much budget you give them Can non-reasoning models catch up with more compute?. Once reasoning models become the default, every question gets more expensive to answer. That is a change on the demand side driven by architecture, not by users.
The same lever can push demand down. Adaptive allocation gives easy prompts less compute and hard prompts more, and it beats bigger models running on a fixed budget Can we allocate inference compute based on prompt difficulty?. Hybrid models like Thinkless learn on their own when extended reasoning is worth it and when a quick answer will do Can models learn when to think versus respond quickly?. Prompt compression can cut inputs by up to 20x with little loss in accuracy Can prompts be compressed without losing model performance?. Some approaches skip visible reasoning altogether: a 150M-parameter model reasoning in its hidden layers costs a fraction of a cent per ARC-AGI task Can latent reasoning match chain-of-thought cost efficiency without verbalizing?. Architecture-aware scaling laws find throughput gains of around 42% at the same training cost Can architecture choices improve inference efficiency without sacrificing accuracy?. Each of these lowers the tokens needed per answer, which makes AI cheap enough for new uses. And by the rebound logic above, that may raise total demand again.
The takeaway you may not have expected: 'how much inference the world needs' is partly a design choice built into models, through how long they think, when they decide to think, and how compact their inputs are. It is not only a market outcome. Price and speed do change demand, but the biggest swings may come from engineers deciding how many tokens a question deserves. What the corpus doesn't have is direct evidence on how end users respond to price changes, so how strong the rebound really is remains an open question.
Sources 9 notes
Denain argues the inference cost burden is not persistent because the dollar cost to reach a given capability level drops rapidly each year through distillation, algorithmic efficiency gains, and hardware advances. He documents a 3x cost reduction over eight months on a math benchmark and projects 5–10x annual reductions, suggesting today's expensive capabilities become affordable within years.
Emberson and Sevilla estimate global token demand at 200 million to 4 billion tokens per second, growing 10× yearly, while inference supply grows only 3.4× yearly due to compute bottlenecks. They cite early signs including Anthropic's quota reductions during peak hours.
Snell et al. (2024) showed that inference-time compute trades off against model parameter scaling, especially on difficult prompts. This reveals pretraining and inference compute are not independent resources.
Reasoning models persistently outperform non-reasoning models regardless of inference budget because training instills a reasoning protocol that makes additional tokens productive. The gap is fundamentally about deployment mechanisms and training structure, not raw capability.
Research shows inference effectiveness varies dramatically by prompt difficulty. Reallocating the same total compute adaptively—giving easy prompts less and hard ones more—substantially outperforms larger models under uniform budgets.
Show all 9 sources
Thinkless trains a single model to select between extended reasoning and direct responses using DeGRPO, which decouples mode selection from answer refinement. This prevents mode collapse and enables self-calibrated routing without explicit difficulty labels.
LLMLingua achieves up to 20x prompt compression with only 1.5-point performance drops by allocating less compression to instructions and questions, more to demonstrations, then pruning tokens using a small model's perplexity scoring aligned to the target LLM.
A 150M-parameter model combining in-context demonstrations with iterative latent computation reached 29.5% pass@2 on ARC-AGI-1 at $0.0007 per task, surpassing previously reported cost-accuracy tradeoffs. The approach separates learning (via demonstrations updating recurrent memory) from reasoning (via iteration in hidden space) without generating intermediate tokens.
Augmenting scaling laws with hidden size, MLP-to-attention ratio, and GQA configuration enables architecture optimization for inference. Optimized models achieved up to 2.1% higher accuracy and 42% greater throughput than LLaMA-3.2 under identical training budgets.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Reasoning Models Can Be Effective Without Thinking
- Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers
- Is a compute crunch coming?
- Rethinking Thinking Tokens: LLMs as Improvement Operators
- When More Thinking Hurts: Overthinking in LLM Test-Time Compute Scaling
- Inference-Aware Prompt Optimization for Aligning Black-Box Large Language Models
- How persistent is the inference cost burden?
- Scaling Laws Meet Model Architecture: Toward Inference-Efficient LLMs