Once an AI agent starts working, does its money go mostly to its own 'thinking,' or to outside tools and services it pays for?
How immediately are AI agents spending on inference versus other services?
This reads the question as asking where an AI agent's spending goes the moment it starts working (on its own model calls, or on outside things like search, tools and paid services), and how quickly that spending piles up.
This explores where an AI agent's money goes once it starts working: on its own thinking (inference), or on outside things like search, tools and paid services. A direct answer first: the corpus has no measurements of agents paying for non-inference services, such as buying data, calling paid APIs or making purchases. What it does show is that inference is the immediate and continuous cost, and that most of the other things an agent does still turn into inference costs.
The basic change is that an agent never stops consuming compute. Training a model is a large cost paid once. A running agent is an inference customer every second it works, and that cost grows with the number of agents and robots deployed, which is why inference is expected to become the main source of compute demand Will inference compute soon exceed training compute demand?. Company budgets already reflect this. Epoch AI found that R&D and inference compute make up 54–62% of operating expenses at Anthropic, Minimax and Z.ai, while staff costs stay below 25% Do compute costs really dominate AI company budgets?.
The surprising part is that "other services" often turn out to be inference wearing a different label. For deep research agents, search steps follow the same scaling curve as reasoning tokens: more searches buy better answers in the same way that more thinking does How does test-time scaling work for individual research agents?. Tool calls work the same way. Every result has to be read back into the model's context and reasoned over. One study found that raw token counts and tool-call counts predict agent performance poorly. What predicts it is how much of that feedback was actually useful and kept for later decisions Does raw token spending actually predict agent performance?. So the useful question about agent spending is how much of it was wasted, more than which category it fell into.
The way spending happens over time is also changing. In a 115-day case study of a long-running agent, 82.9% of tokens were cache reads, meaning old context being reused rather than new thinking. At that point the sensible way to count cost is per finished piece of work, not per token Do persistent agents really cost less per token?. "Sleep-time compute" pushes this further: the agent does its thinking about stable context during idle time, before any question arrives, so less of the cost lands at the moment someone is waiting When should AI systems do their thinking?. Agents can also cut costs by spending more compute only on hard prompts Can we allocate inference compute based on prompt difficulty?, or by learning when to skip extended thinking altogether Can models learn when to think versus respond quickly?. Over a long multi-step task these savings add up, and a smaller model trained for follow-through can beat a larger one on cost Does model efficiency matter more than peak capability for real work?.
The takeaway: inference is the cost that starts immediately and never stops, and search and tools mostly feed into it rather than competing with it. The interesting work is now about moving that spending in time, by caching, precomputing and routing, rather than simply reducing it. If you meant agents spending money on outside goods and services, this part of the corpus doesn't cover it. That material is more likely filed under agentic commerce.
Sources 9 notes
A seven-layer AI compute stack model reveals that inference demand grows along a new axis training never had: every autonomous agent is a continuous inference consumer. This makes inference, not training, the likely dominant compute demand as agents and robots scale.
Epoch AI's analysis of Anthropic, Minimax, and Z.ai shows R&D and inference compute account for 54–62% of total operating expenses, while staffing remains below 25% despite high salaries.
Research shows that deep research agents exhibit test-time scaling laws where search steps scale similarly to reasoning tokens, and live search outperforms memorized retrieval on knowledge-intensive tasks. Data efficiency is extreme—78 curated demonstrations outperform 10K samples for agency.
Effective Feedback Compute—crediting only informative, valid, non-redundant feedback retained for decisions—predicts performance (R²≈0.94) far better than raw tokens or tool calls (R²≈0.33–0.42). The scaling lever is feedback quality, not quantity of interaction.
A 115-day case study found 82.9% of tokens were cache reads. When context persists and reuses, the meaningful cost denominator becomes completed artifacts, not individual tokens.
Show all 9 sources
Rather than bundling context and queries into one inference call, models can precompute reasoning over stable context during idle periods, then answer queries efficiently. This reduces latency, amortizes cost, and enables more sophisticated reasoning than query-time budgets allow.
Research shows inference effectiveness varies dramatically by prompt difficulty. Reallocating the same total compute adaptively—giving easy prompts less and hard ones more—substantially outperforms larger models under uniform budgets.
Thinkless trains a single model to select between extended reasoning and direct responses using DeGRPO, which decouples mode selection from answer refinement. This prevents mode collapse and enables self-calibrated routing without explicit difficulty labels.
Occamy-1.0, a 35B-parameter model further trained on execution-grounded data and long-horizon trajectories, achieves competitive performance with much larger models while sitting at the low-cost knee of the Pareto frontier, suggesting that training for coordination and follow-through substitutes for raw scale in multi-step work.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Reasoning Models Can Be Effective Without Thinking
- How we built our multi-agent research system
- The Price of Progress: Price Performance and the Future of AI
- Is a compute crunch coming?
- When More Thinking Hurts: Overthinking in LLM Test-Time Compute Scaling
- RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts
- Thinkless: LLM Learns When to Think
- The plunging price of thought