When a company decides what to automate, do managers ever check whether the AI messes up on the same cases people do?

This explores whether managers, when deciding what to automate, check how far the AI's mistakes overlap with their people's mistakes, or whether they only compare who is more accurate. The short answer is that the corpus doesn't document what managers actually do. No note here surveys or observes real automation decisions. What the corpus does show is why error overlap is the measure that should matter, and why ordinary evaluation habits tend to miss it.

The central finding is counterintuitive. In a team that combines human and AI predictions, the person most at risk of being replaced is not the least accurate one. It's the one whose errors most closely match the AI's Does error diversity matter more than accuracy for human replacement?. A person who is less accurate but wrong in different places adds information the AI lacks. A person who is accurate but wrong in the same places adds almost nothing. So a manager who ranks staff by individual accuracy, which is the obvious metric, could keep the wrong people and automate away the very coverage that catches the AI's blind spots.

There are signs this blind spot is structural rather than accidental. Safety failures in deployed AI tend to be plausible, spread across many small cases, and absorbed into normal workflows. They aren't dramatic or easy to spot, so the instruments organizations use are tuned to the wrong kind of failure Why do safety failures remain invisible to our evaluation methods?. Shared errors are a clear example. They don't show up in accuracy dashboards, and they only become visible when the human and the AI fail on the same case together. Greater automation also tends to produce polished outputs that hide errors instead of removing them Does more automation actually hide rather than eliminate errors?. That makes it even less likely anyone notices that the human backup is failing in the same places.

The automation literature shows what happens when this goes unmeasured. A meta-analysis of 18 experiments found automation improves routine performance but sharply degrades recovery when it fails, especially once it moves from analyzing information to choosing actions When does automation harm performance after failures?. If the operator's judgment has also drifted toward the system's own patterns, the human "fallback" is weaker than it looks on paper. When that failure arrives, blame tends to land on the operator anyway Do automated systems shift blame onto their human operators?.

A useful reframe runs through several notes: treat automation as a question of how to design the whole system, not as a task-by-task swap. AI can raise individual task performance while increasing variation across decisions, which forces organizations to rethink how they coordinate Does AI adoption require rethinking how organizations coordinate?. Automation and human control can also be designed as independent dimensions rather than a trade-off Can automation and human control increase together?. Read together, these suggest the right question isn't "is the AI better than this person?" It's "does this person catch what the AI misses?" That is a different measurement, and the corpus gives little sign that it is routine practice.