AI's token use is exploding fast — but is that from longer conversations, or just way more people using it?
How much of token demand growth comes from context length versus user volume?
This explores whether the fast growth in AI token consumption comes mainly from each request getting longer (bigger contexts, longer agent runs) or from more people and more requests, and what the corpus can say about that split.
This explores whether token demand is growing because each AI interaction uses more tokens, or because more people are making more requests. The short answer: the corpus does not split the growth into those two parts. What it does have is a headline growth figure plus several case studies showing how fast tokens per task can grow, and that is where the interesting part is. Will AI inference demand outpace supply capacity soon? estimate global demand at roughly 200 million to 4 billion tokens per second, growing about 10× a year, while inference supply grows only about 3.4× a year. The size of that range (a factor of 20) tells you how rough these estimates still are. Nobody here has divided the 10× into a share from new users and a share from longer interactions.
The per-interaction side is where the corpus has the most to say, and it points to agents more than chat. An agent doesn't just send a longer prompt. It re-reads its whole working context on every step. A 115-day study of a persistent agent found that 82.9% of its tokens were cache reads, meaning the same context was processed over and over Do persistent agents really cost less per token?. Anthropic's internal evaluations found that token spending alone explains about 80% of the performance differences between multi-agent research systems Does token spending drive multi-agent research performance?. Deep-research agents follow a scaling curve where more search rounds keep buying better answers, with diminishing returns Does search budget scale like reasoning tokens for answer quality?. Taken together, one user running one agent can consume what used to take hundreds of chat sessions. So 'context length versus user volume' may be the wrong way to cut it: the bigger driver is probably the number of steps per task, which is neither.
Some of that growth isn't real need. Meta's internal leaderboard ranking employees by token use led people to leave agents running idle and pad their contexts Does token volume measure AI productivity or enable gaming?. When token volume becomes a status metric, demand figures partly measure incentives, not usefulness. This matters for the headline numbers, because organizational mandates to 'use AI more' can inflate per-user consumption without anyone getting more done.
The flip side is that per-task token counts can shrink a lot, which makes this kind of demand more elastic than user growth. Automatically optimized agent setups cut token traffic by 45–49% with about the same performance Can agent harnesses be automatically optimized across many environments?. Prompt compression reaches up to 20× with small accuracy losses Can prompts be compressed without losing model performance?. Retrieving 8 of 32 user-profile fields matched the full profile at about a quarter of the tokens Can a smaller user model subset match full model performance?. Agent performance tracks useful feedback far better than raw token count Does raw token spending actually predict agent performance?. So much of the per-task token growth is waste that better engineering could remove. Growth from more users can't be engineered away like that.
The point worth taking away: if the coming compute crunch is driven mostly by bloated agent loops rather than by more people, efficiency work could do far more to close the supply gap than building more data centers. The corpus can't yet tell you which side dominates. Someone would need a study that breaks token growth down by users, requests per user, and tokens per request.
Sources 9 notes
Emberson and Sevilla estimate global token demand at 200 million to 4 billion tokens per second, growing 10× yearly, while inference supply grows only 3.4× yearly due to compute bottlenecks. They cite early signs including Anthropic's quota reductions during peak hours.
A 115-day case study found 82.9% of tokens were cache reads. When context persists and reuses, the meaningful cost denominator becomes completed artifacts, not individual tokens.
Anthropic's internal evals show token spending alone accounts for 80% of performance variance in multi-agent research systems. Model capability upgrades deliver larger gains than doubling token budget, suggesting efficiency matters as much as quantity.
Agentic deep research shows monotonic-to-diminishing-returns curves for search iterations, matching reasoning token scaling. This creates a new inference-compute axis: models can trade off reasoning budget against search budget to optimize answer quality.
Meta built an internal leaderboard ranking 85k+ employees by AI token usage with tiered badges. Employees respond by running idle agents for hours and padding contexts unnecessarily—gaming behavior the note documents—yet the company provides no outcome data linking token volume to actual productivity or business results.
Show all 9 sources
Cross-environment harness optimization yielded four mechanisms (action execution, context compaction, observation handling, delegated reading) that reduced token traffic by 44.7–49.0% while maintaining comparable performance on a 51-task benchmark, suggesting harness-level gains are orthogonal to model improvements.
LLMLingua achieves up to 20x prompt compression with only 1.5-point performance drops by allocating less compression to instructions and questions, more to demonstrations, then pruning tokens using a small model's perplexity scoring aligned to the target LLM.
Researchers found that retrieving only 8 of 32 user model fields achieved equivalent personalization performance while using 211 tokens instead of 915. This enables bounded disclosure per query while keeping sensitive fields local.
Effective Feedback Compute—crediting only informative, valid, non-redundant feedback retained for decisions—predicts performance (R²≈0.94) far better than raw tokens or tool calls (R²≈0.33–0.42). The scaling lever is feedback quality, not quantity of interaction.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- How we built our multi-agent research system
- Is a compute crunch coming?
- Single-Agent LLMs Outperform Multi-Agent Systems on Multi-Hop Reasoning Under Equal Thinking Token Budgets
- Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent Evaluation
- RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts
- Scaling Laws for Agent Harnesses via Effective Feedback Compute
- When More Thinking Hurts: Overthinking in LLM Test-Time Compute Scaling
- The Price of Progress: Price Performance and the Future of AI