Why does letting an AI decide its own next move cost so much more than just following a fixed script?

This explores why an AI agent that decides its own next steps costs more in time and compute than a workflow, where developers set the sequence of LLM calls ahead of time. The basic answer is structural. A workflow makes a fixed number of model calls. An agent runs a loop: it looks at where things stand, decides what to do next, calls a tool, reads the result and decides again. Each turn is another model call, and each call carries the growing history of everything so far. Anthropic's guidance puts it plainly: agents trade latency and cost for flexibility. That trade only pays off when the path through a task can't be predicted in advance, so the advice is to start with the simplest single LLM call, move to a workflow when the task breaks into known steps, and use agents only after that (When should teams choose workflows over agents?).

The less obvious part is how much of an agent's apparent intelligence is really just spending. Research on multi-agent systems finds that about 80% of the variation in their performance comes from how many tokens they use, not from smarter coordination (How does test-time scaling work at the agent level?). So part of the extra cost is not overhead around the work. In many setups, the extra tokens are where the gains come from. This is why that line of research looks for ways to share memory between agents (such as a shared cache of already-processed context), so the gains don't scale one-for-one with the bill. Coordination adds its own cost. When many agents must agree, they often agree too late or pass along unchecked information, which means more rounds of messages and more chances for errors to spread (Why do multi-agent systems fail to coordinate at scale?).

Some of the cost is self-inflicted. Agent benchmarks that score only accuracy push developers toward more and more elaborate agents, because nothing in the score punishes waste. Treating cost and accuracy as a trade-off to optimize together often finds designs that are just as accurate and much cheaper (Why do accuracy-only benchmarks hide the true cost of agents?). How an agent acts on the world matters too. Agents that click through user interfaces one screen at a time are slow by design. Calling an application's API directly instead cut task completion time by 65–70% with almost no loss in accuracy (Can API-first agents outperform UI-based agent interaction?).

The cost gap is not fixed, and the corpus points to several ways to narrow it. Most agent subtasks are routine language jobs that small models can handle at 10–30× lower cost, so a mixed design (small models by default, large ones only when needed) is the cheaper pattern (Can small language models handle most agent tasks?). Because cost and delay pile up over a whole multi-step episode, a mid-sized model trained on real execution traces can sit at the best point on the cost–performance curve, ahead of much larger models (Does model efficiency matter more than peak capability for real work?).

Here's the reframe you might not expect. In a 115-day study of a long-running agent setup, 82.9% of tokens were cheap cache reads, meaning context the agent had already processed and simply reused (Do persistent agents really cost less per token?). The authors argue that cost per token becomes the wrong measure. The better question is what each finished piece of work costs. Seen that way, an agent's higher per-run cost can be the price of finishing tasks that no fixed workflow could complete. One caveat: the corpus is stronger on why the overhead exists and how to cut it than on direct measurements comparing agents and workflows on the same task.