When does it actually pay off to build an AI that plans and acts on its own, instead of something simpler?
When should designers choose agentic systems over simpler alternatives?
This explores when an AI system that plans and acts on its own is actually worth building, compared with a single LLM call or a fixed, scripted workflow, and what the corpus says about the hidden costs of choosing "agentic" too early.
This explores when an AI system that plans and acts on its own is worth building, compared with a single model call or a fixed, scripted workflow. The clearest rule in the corpus comes from Anthropic: start with the simplest LLM call, move to a workflow when a task splits cleanly into known steps, and use an agent only when you can't map out the path to the answer ahead of time When should teams choose workflows over agents?. You pay for agents in latency, cost and unpredictability. They only earn that price when a task needs decisions you couldn't have scripted in advance.
It helps to treat "agentic" as a ladder rather than a yes/no switch. Dubberly lays capabilities out along two axes. One climbs from a bare chatbot to a system prompt to retrieval (RAG), which adds inputs. The other climbs from monitoring agents to supervisors to orchestrators, which adds layers of oversight How should agent capabilities be organized into functional layers?. Many problems that seem to call for an agent are solved one rung up, with better context or retrieval, and never need autonomy. Dubberly also warns that stacking agents on top of agents brings design risks along with the extra capability.
The corpus is surprisingly unsure about multi-agent setups. One line of work argues that single agents hit a limit no amount of extra capability can fix: tasks that need several kinds of expertise, parallel work and independent checking need a team of specialized agents Do single agents always hit organizational limits?. Cognition's Walden Yan argues close to the opposite. Parallel subagents each make unspoken assumptions the others can't see, so their results clash, and the safer default is one agent working step by step with continuous context Why do parallel subagents produce conflicting design decisions?. Read together, these suggest that splitting work across agents helps when subtasks are truly independent and can be checked separately, and hurts when they share hidden design decisions. Older multi-agent research adds a related warning: if you don't explicitly model the environment the agents work in, what you designed and what actually runs drift apart Should multiagent systems model their environment explicitly?.
Another lesson that's easy to miss: when an agent does work, the reliability often comes from the structure around the model, not from the model's own judgment. Dependable agents hand off memory, reusable skills and interaction protocols to a surrounding "harness" so the model doesn't have to solve the same problems every time Where does agent reliability actually come from?. The same goes for the environment. Changing interfaces to suit how agents perceive and click improves task completion without making things worse for human users Can interface design help agents complete computer tasks better?. So choosing an agent also commits you to building scaffolding. The organizational version of this point comes from Deloitte, which finds that agents dropped onto old workflows tend to fail, while redesigning the whole process around them succeeds Does agentic AI succeed through process redesign or workflow layering?.
Finally, the stakes of the domain should shape the choice. A review of nine deployed agents found that general productivity tools followed good interaction principles much better than companion or high-stakes advisory agents. Those were weakest at preventing user dependency and at making their intentions visible Do productivity agents actually protect users better than emotional ones?. Governance raises a similar flag: the usual tools for keeping a delegate in line (incentives, monitoring, enforcement) break down when an agent's decisions are fast and hard to interpret Can traditional governance tools actually control AI agents?. A practical test follows from this. Choose an agent when the path can't be scripted, the task is worth the cost, you're prepared to build the harness and redesign the workflow, and the domain can tolerate decisions you can't fully inspect. If any of those fails, a simpler system is probably the better design.
Sources 10 notes
Anthropic recommends a simplicity-first discipline: optimize single LLM calls and workflows for decomposable tasks before adopting agents, since agents trade latency and cost for flexibility only when task paths cannot be predetermined.
Dubberly proposes a two-axis stack model: a capability axis (chatbot → system prompt → RAG) adding inputs that extend what LLMs can act on, and a scale axis (monitoring agents → supervisors → orchestrator) aggregating oversight. He cautions that ubiquitous agent hierarchies raise design risks alongside capability gains.
Research shows that real-world tasks requiring heterogeneous expertise, parallel execution, and independent verification exceed what any single agent loop can organize. Graph-based system abstractions are needed to distribute intelligence across specialized agents.
Walden Yan argues multi-agent parallelism is fundamentally fragile: subagents embed unwritten assumptions in every action, so agents working independently converge on conflicting choices. The fix is reverting to single-threaded linear agents where context remains continuous.
Gaia's 2003 methodology argues that MAS designers must model environments (physical or virtual) as primary abstractions during analysis and design. Skipping this step causes complexity and misalignment between intended design and actual operation, as illustrated by a conference-management system example.
Show all 10 sources
Research shows reliable LLM agents externalize three cognitive burdens—memory (state persistence), skills (procedural components), and protocols (structured interaction)—into a harness layer rather than relying on model scale alone. The harness unifies these externalities and eliminates the need for the model to solve the same problems repeatedly.
UI-Verse benchmark testing found that augmented heuristics targeting agent perception, grounding, and control consistently improved agent task completion and efficiency. Human studies confirmed these designs preserved usability for human users with no observable regressions.
Deloitte argues many agentic AI implementations fail because organizations layer agents onto old workflows rather than redesigning operations. Success requires agent-compatible data architectures, robust orchestration, and new management models; HPE and Toyota illustrate this approach.
A heuristic evaluation of nine deployed agents found that general-purpose productivity systems (ChatGPT, Claude) scored substantially higher on 14 HCI principles than emotionally sensitive or high-stakes agents (Character.ai, Consensus, Pine AI). Risk-critical principles like dependency prevention and intent visibility were most violated in companion and advisory contexts.
Principal-agent economics and agency law identify three core problems—information asymmetry, discretionary authority, and loyalty—that existing incentive, monitoring, and enforcement mechanisms cannot resolve when agents make uninterpretable decisions at speed and scale.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Drop the Hierarchy and Roles: How Self-Organizing LLM Agents Outperform Designed Structures
- Towards a Science of AI Agent Reliability
- Why Do Multi-agent LLM Systems Fail?
- Design Principles for Human-Agent Interaction
- Towards a Science of Scaling Agent Systems
- Explaining AI Agents Through Execution Traces
- A Framework for Studying AI Agent Behavior: Evidence from Consumer Choice Experiments
- Institutional AI: Governing LLM Collusion in Multi-Agent Cournot Markets via Public Governance Graphs