When does automation harm performance after failures?

Onnasch, Wickens, Li, and Manzey's 2014 meta-analysis of 18 experiments tests how degree of automation (DOA) — defined by the Parasuraman-Sheridan-Wickens stages-and-levels taxonomy (Stage 1: information acquisition, Stage 2: information analysis, Stage 3: decision and action selection, Stage 4: action implementation) — affects routine performance, failure performance, workload, and situation awareness (SA). The excerpt reports "a clear automation benefit for routine system performance with increasing DOA," "a similar but weaker pattern for workload," and "a negative impact of higher DOA on failure system performance and SA." The paper's most specific finding, stated in its conclusion, is that "negative consequences of automation are most likely when DOA moves from Stage 2 to Stage 3 automation" — the boundary between automation that analyzes information and automation that selects or takes action.

The authors frame this as the "lumberjack effect": "the higher they are, the farther they fall" — more automation improves routine performance and lowers workload but increases the cost when automation fails, because operators monitor less, trust more, and lose SA. Crossing into Stage 3 is where this cost concentrates because removing the human's role in choosing an action removes what the authors call "the generation effect" (citing Slamecka & Graf, 1978): "actively choosing actions manually... supports SA in a way that supports the manual performance in case of automation breakdown. When that choice is removed, the automation failure response suffers." The paper explicitly rejects "medium DOA is always best" as a design rule, since the trade-off is continuous and direct — "each step of increase of DOA seems to be associated with an increase of the risk of return-to-manual performance decrements" — but narrows the recommendation to: if return-to-manual performance matters, keep humans involved specifically in decision/action-selection and implementation, not merely in the information stages.

This gives a pre-LLM, quantitative precedent for claims now made about AI agents. Does granting agents more autonomy undermine human oversight? argues autonomy and extended use both erode oversight; this meta-analysis locates exactly where in a human-automation system that erosion concentrates — not uniformly with "more automation" but specifically at the move from analysis to action selection. It likewise sharpens Does AI augmentation protect workers from skill erosion?: the generation-effect mechanism explains why automation framed as choosing actions for a human, rather than merely surfacing information, is the design move most likely to erode skill and SA. And it offers an empirical echo of Does targeted human oversight beat both full autonomy and exhaustive review?, since both findings put the leverage point at decision/action stages rather than treating all human involvement as equally protective.

The excerpt is explicit about scope limits: DOA is only an ordinal ranking built from 18 studies across varied domains (aviation, process control, and similar), not an interval or ratio measure, so the paper cannot quantify how much risk each step of DOA adds, and it says nothing about LLM-based or generative AI agents, which did not exist in the automation literature this 2014 paper synthesizes. The finding also concerns the SA and skill costs of automation that works and then fails, not the separate question of oversight on systems that are unreliable from the start. The implication for agentic-AI design, at the strength this evidence allows, is that where an organization places the human in a pipeline — before versus after the system selects an action — likely matters more for retained oversight capacity than how much automation exists overall.