Chatbots are repeating the same automation mistakes aviation and factory engineers spotted decades ago — what are they?

This explores the four automation failures that Simkute and colleagues say generative AI repeats, and why lessons from decades-old aviation and control-room research apply to chatbots. Their argument is that GenAI's uneven productivity gains aren't new. They're the same 'ironies of automation' researchers documented long before LLMs existed Does generative AI productivity actually improve or decline in practice?. The four are: (1) **your role shifts from doing to evaluating**, so you stop writing and start checking the machine's output, which is a different and often harder skill; (2) **workflows get disrupted**, because the tool doesn't fit how the work actually flows; (3) **interruptions break concentration**, because prompting, waiting and correcting fragment the focus that hard work needs; and (4) **easy tasks get easier while hard ones stay hard**, so the help lands where it's needed least.

The first irony is the one the rest of the collection backs most clearly. In interviews, 18 lawyers described GenAI summaries that looked efficient but took longer to re-verify than doing the work by hand. The cause wasn't mainly errors. It was opacity: the lawyers couldn't see where claims came from, yet they were still accountable for them Does GenAI actually save lawyers time on fact verification?. That is the shift to evaluation in practice. Checking a fluent answer you can't trace can cost more than producing it yourself.

The less obvious point is why evaluation gets harder over time, not just more tedious. A meta-analysis of 18 pre-GenAI experiments found that automation improves routine performance but weakens people's ability to recover when it fails. The drop is sharpest when the automation moves from analyzing information to choosing actions, because the people watching it lose both the skill and the awareness they need to take over When does automation harm performance after failures?. Other notes describe the same pattern with modern AI. Polished outputs hide errors instead of removing them Does more automation actually hide rather than eliminate errors?, and the most dangerous systems are the ones that seem competent while quietly wearing down the skepticism that oversight depends on How do competent systems quietly undermine safety oversight?. So the irony builds on itself: the better the tool looks, the worse people get at catching its mistakes.

The skill side is more mixed than 'AI deskills workers.' A study of 38 knowledge workers found four different outcomes: skills developed, maintained, eroded, or revalued. Which one happened depended on how each person used the tool and which tasks changed How does generative AI actually change worker skills?. That fits the fourth irony: if AI mostly absorbs easy work, whether your skills grow or erode depends on what you do with the hard work that's left. At the economy-wide level, the ironies may help explain why measured gains are small. Survey data puts generative AI's current aggregate productivity gain at about 1.1% What is generative AI's actual aggregate productivity gain today?.

The same authors also borrow the fixes. They map five human-factors principles from automation design onto GenAI: continuous feedback, personalization, interfaces that show the system's state, task stabilization, and clear division of who does what Can Human Factors design fixes restore GenAI productivity losses?. These are untested proposals, not proven remedies. Still, the lesson matters: these failures come from how humans and machines share the work, so better models alone probably won't fix them.