Do automated systems shift blame onto their human operators?

Elish introduces "the concept of a moral crumple zone to describe how responsibility for an action may be misattributed to a human actor who had limited control over the behavior of an automated or autonomous system." The image is deliberate: "Just as the crumple zone in a car is designed to absorb the force of impact in a crash, the human in a highly complex and automated system may become simply a component... that bears the brunt of the moral and legal responsibilities when the overall system malfunctions." The asymmetry with the car is the point of the essay: "While the crumple zone in a car is meant to protect the human driver, the moral crumple zone protects the integrity of the technological system, at the expense of the nearest human operator." Elish opens with the 2018 Uber self-driving car fatality in Arizona, where the safety driver "may face criminal charges of vehicular manslaughter" despite a job limited to monitoring and emergency takeover.

Elish builds the concept from three cases chosen because operators sat in different relationships to the malfunction, yet all three absorbed blame. At Three Mile Island, "the operators knew the system was malfunctioning, but they did not have sufficient information or authority to take corrective actions" — the control room gave no direct reading of cooling-water level, only an auxiliary tank gauge that read full and misled them. In the Therac-25 radiation-overdose cases, "the operator had no way of knowing that the system had malfunctioned, except for reports from patients that felt pain," and the harm surfaced only after "months of inquiries" had already assigned fault to the manufacturer. On Air France 447, the pilots "had sufficient information to react" but were, in Elish's phrase, "systemically disempowered to act appropriately on that information." Media coverage and official reports (she cites the Kemeny Commission on TMI) then settle blame on the operator even while naming "systemic" causes, because operators become, in her term, "liability sponges": "the technology is maintained as faultless, while the human operator becomes the faulty feature of the system."

This sits beside rather than inside the vault's oversight-capacity thread. Can organizations lose scrutiny capacity while keeping oversight forms? and Does granting agents more autonomy undermine human oversight? both describe a human reviewer losing the practical ability to scrutinize a system in real time, before any accident occurs. Elish's cases run the other direction: her operators sometimes had partial information and partial authority during the event, but the moral crumple zone is assigned afterward, by investigators and media, regardless of how much capacity the operator actually had. The two phenomena compound — an operator whose scrutiny capacity has already eroded is also the person best positioned to become the post-hoc scapegoat — but nominal-oversight erosion is a design and organizational condition, where moral crumple zones are an attribution dynamic performed by commentators, regulators and the press after something breaks. The pattern also parallels Does the UN panel misframe the OpenAI breach as alignment?, where a governance body's framing choice similarly moves accountability away from the organization that built or maintained the system.

What the excerpt does not establish: it names no rate at which moral crumple zones form, no count of cases beyond the ones discussed, and makes no claim about LLM agents, which did not exist when Elish wrote in 2019. Her argument is conceptual and historical — TMI (1979), Air France 447, Therac-25, and one 2018 self-driving case — offered as a lens, not a measured finding. The implication she draws is for certification and regulatory design: if responsibility is assigned by looking at who was nearest the failure rather than who held the matching control and information, oversight structures for agentic systems risk manufacturing a new class of liability sponges rather than new accountability.