Does increased automation erode human oversight ability?

The same paper's "Related work" section carries a second, separable claim. Lisanne Bainbridge's 1983 paper "Ironies of Automation" identified recurring costs of increasing automation in process control and aviation, and the excerpt names three. The vigilance problem: "As the degree of automation increases, humans increasingly take on the task of vigilance, becoming monitors of what the system is doing rather than active participants," and classic vigilance studies found "even highly motivated subjects found it difficult to maintain attention for more than about half an hour." The takeover challenge: when an automated system hands control back, "human operators have been paying attention elsewhere, meaning they have reduced situational awareness," a failure mode the excerpt illustrates with China Airlines Flight 006 in 1985, where "reliance on the 747's autopilot during an engine failure introduced complexities contributing to disorientation of the pilot, leading to a sudden uncontrolled 30,000 foot plunge." De-skilling: Bainbridge "drew attention to the irony of not only physical skills deteriorating, but also cognitive skills waning when processes are increasingly automated," which matters because "when automation fails, it is just these skills that are crucial when the human needs to step in."

Bainbridge's own framing is the irony itself: automation is introduced to relieve humans of a task, but it leaves them with the hardest version of that task — rare, high-stakes intervention — while removing the routine practice that would keep them able to perform it. The paper extrapolates this 1983 aviation and process-control argument to generative AI by analogy, asking "what AI applications mean for a world in which many of our jobs might increasingly require us to monitor or oversee what our intelligent systems are doing" and whether "our jobs will become more stressful and less satisfying as a result" — framed explicitly as an open question, not a measured outcome.

This supplies the foundational source behind two claims already in the library. Does granting agents more autonomy undermine human oversight? and Does AI augmentation protect workers from skill erosion? both argue, from recent agent-specific papers, that oversight capacity erodes with use — this note names Bainbridge's 1983 vigilance/takeover/de-skilling triad, and aviation's historical record, as the lineage those claims trace back to, decades before LLM agents existed. Does targeted human oversight beat both full autonomy and exhaustive review? reads as a partial answer to the takeover challenge: confidence-routed handoff is one way to narrow the window in which a disengaged human must suddenly re-engage.

Bainbridge wrote about industrial process control and flight decks in 1983, and the China Airlines incident is a single historical case from aviation, not a study of AI agents or of generative AI users. The paper's extension of this argument to AI is explicitly stated as an open question and an extrapolation, not a finding — it reports no measurement of vigilance decrement, takeover difficulty, or de-skilling among people working with AI systems. The implication that follows is correspondingly modest: the aviation record makes the oversight-erosion concern plausible and gives it a name and a mechanism, but establishing that it applies to AI agents is work the more recent, agent-specific papers already in the library have started to do, not something this source itself measures.