Should you build a flexible AI agent or a simple script? It depends on how messy and unpredictable the task really is.

This explores how to choose between an AI agent and simpler automation, based on how unpredictable a task is and how standardized the process already is. The clearest guidance in the corpus comes from McKinsey's framework When should you actually deploy an AI agent instead of simpler automation?. It says to start from the shape of the work, not from what agents can do. Agents earn their place on high-variance, low-standardization tasks, such as pulling data out of financial documents that are each laid out differently. Predictable, well-governed workflows are better served by rules-based or scripted automation. If you default to agents anyway, you add complexity without getting matching value back.

Other parts of the corpus suggest why this rule holds up in practice. Teams putting agents into production keep removing flexibility rather than adding it. One case study found that letting an agent discover and choose tools through a protocol layer (MCP) caused unpredictable failures. The agent picked the wrong tool or guessed parameters incorrectly. Reliability came back only after the team switched to explicit function calls, with one tool per agent Why do protocol-based tool integrations fail in production workflows?. The same note cites a survey of 306 practitioners: 85% build custom agents instead of using general frameworks. In other words, even when a team does choose an agent, it often makes the standardized parts of the job deterministic again.

The hidden cost of choosing wrong is that agent failures can be hard to see. When models edit documents across a long workflow, weaker models tend to delete content, which is easy to spot. Stronger models are more likely to corrupt content quietly while the document still looks intact Does model capability change how documents degrade?. On a standardized task, a simple script either works or fails visibly. An agent on that same task brings in a failure mode that gets subtler as the model gets better, which strengthens the case for keeping predictable work on simpler automation.

When variance really is high enough to justify agents, the corpus suggests putting effort into the structure around the model. Agent-based evaluators that gather evidence step by step were about 100 times more consistent than a single LLM judge. Yet one of their modules (memory) passed its errors on to later steps Can agents evaluate AI outputs more reliably than language models?. Separately, automatically tuning the surrounding software (the harness: how actions are run, how context is compressed, how outputs are read) cut token use by almost half without making results worse Can agent harnesses be automatically optimized across many environments?. So matching automation to variance doesn't end once you've picked a type. You also standardize every part of the agent's work that can be standardized and keep its open-ended reasoning for the parts that truly need it.

One caveat: the corpus has only one note that addresses this decision framework directly. Most of what's above is supporting evidence from production and evaluation research, not a tested method for choosing an automation type.