Automation looks great on a normal day — but the real test is how badly things go when something breaks.

This explores how designers can first find out how much automation a system can safely take on, judged mainly by what it does to the people working with it, before narrowing the choice with secondary factors like cost or workflow fit. The corpus doesn't lay out a step-by-step testing protocol. It does make one point very clearly: if you only test automation when things are running normally, you'll pick the wrong range, because the costs of automation stay hidden until something breaks.

The strongest evidence comes from a meta-analysis of 18 experiments. Automation reliably improves routine performance. But it makes recovery from failures worse and weakens operators' awareness of what's going on. The damage is sharpest when automation moves from *analyzing information* to *choosing actions* When does automation harm performance after failures?. So the useful test isn't "does the task go faster?" It's "when the automation fails partway through, can the person take over?" Testing for an acceptable range means deliberately injecting failures, and testing each stage of automation separately rather than treating 'more automated' as one dial. A related warning: more automation tends to produce polished output that hides errors rather than removing them Does more automation actually hide rather than eliminate errors?. A clean test run can therefore be misleading in its own right.

The second lesson is what to measure. Checking each component on its own isn't enough. Every step can pass its local check while the overall workflow still fails, because local checks test different properties than end-to-end safety requires Can individual components pass safety checks if the system still fails?. Agent evaluation is moving the same way, scoring whole interaction sequences for recoverability and robustness instead of only checking the final answer How should we evaluate agent behavior beyond final answers?. There's a catch: nobody yet has a single tool that measures whether errors stay *visible*, *contained* and *recoverable* across the whole human-plus-system setup. Only partial measures exist How can we measure whether AI errors stay visible and recoverable?.

Once you know which range is acceptable, secondary factors help narrow it. One practical factor is how variable the work is. Predictable, standardized tasks suit simpler automation. Agents earn their added complexity only on messy, high-variance work When should you actually deploy an AI agent instead of simpler automation?. A useful reframe is that automation and human control don't have to trade off: they can be designed as two separate dials, both turned high Can automation and human control increase together?. That makes the range wider than a single scale suggests. Human-factors principles such as continuous feedback and clear task allocation have been mapped onto generative AI design, but they remain untested proposals Can Human Factors design fixes restore GenAI productivity losses?.

The insight you might not expect is that the right test window is years, not sessions. If you automate entry-level work, you remove the apprenticeship that produces the experts who will later need to check the automation How does automating junior work destroy verification oversight?. A level that looks acceptable in a lab study can slowly wear away the human oversight it depends on. Any honest test of an acceptable range has to ask who will be able to take over five years from now.