Is there one dial for 'how automated' something is, or does that number hide more than it tells you?
How do we measure stages of automation consistently across different domains?
This explores whether there's a common yardstick for how automated something is, one that works the same way for a cockpit, a job, an AI agent, or AI research itself, and what the corpus says such a yardstick would need to capture.
This explores whether there's one consistent way to say how automated something is, whether that's a control room, an occupation, or an AI agent. The corpus doesn't offer a single scale. What it does offer is the reason a single scale fails, plus a framework that carries across domains better than most. Parasuraman, Sheridan and Wickens argue that automation isn't one dial. It's four: gathering information, analyzing it, choosing what to do, and carrying out the action. Each can be set to its own level Should each automated function have its own independent level?. A system can be heavily automated at sensing and barely automated at deciding. Calling it "Level 3" overall hides exactly the thing you most need to know.
Those four parts aren't equally risky. A meta-analysis of 18 experiments found that automation helps routine performance but makes recovery from failures worse. The sharpest drop comes at one specific step: when automation stops analyzing information and starts choosing actions When does automation harm performance after failures?. So the most useful measure, in any domain, may not be "how much is automated." It may be "has the system crossed from advising to deciding?" That crossing also tends to shift blame. When systems fail, responsibility lands on the nearby human operator even if they had little real control, which Elish calls a "moral crumple zone" Do automated systems shift blame onto their human operators?. A consistent measure of automation would track who holds authority, not just who does the work.
In economics, automation is usually measured a different way: as the share of a job's tasks that could be automated. McKinsey uses this approach to sort occupations into people-centric, agent-centric, robot-centric, and hybrid groups How do occupations differ in their exposure to automation?. Gans and Goldfarb show why task share can mislead. In many jobs, tasks depend on each other, so automating some of them makes the remaining human work more valuable, not less Does automating some tasks actually increase the value of remaining work?. Task share can also miss effects that build up over time. Automating junior work removes the training ground where future experts learn to check AI output How does automating junior work destroy verification oversight?. A snapshot measure can't see that loss coming.
For AI agents, the measurement problem shows up as evaluation design. Several notes argue that a pass/fail score on the final answer isn't enough. You have to look at the whole sequence of steps: how the agent recovered from mistakes, coordinated with tools, and spent resources How should we evaluate agent behavior beyond final answers? How should we measure agent system performance beyond task success?. Automated benchmarks can both overstate and understate what agents can actually do on messy real-world tasks Do automated benchmarks hide what frontier AI systems can really do?. One tool, BenchShield, replaces the final score with recorded evidence of how the task was completed Can infrastructure evidence replace terminal scores in benchmark validation?. Forecasts of fully automated AI research take a similar route. Kwa's simple model matches far more complex ones by tracking concrete capability measurements instead of vague assumptions Can simpler models predict AI R&D automation timelines accurately?.
The corpus is candid that nobody has built the unified instrument yet. One note lists partial measures: whether errors stay visible, whether they're contained, and how quickly they can be rolled back. It finds that none of these connects the pieces or captures the human and institutional side How can we measure whether AI errors stay visible and recoverable?. The most consistent measure the collection points to would combine the four-function breakdown with two further questions: has the system moved from advising to deciding, and can humans still see and fix its mistakes when it does?
Sources 12 notes
Parasuraman, Sheridan and Wickens (2000) show that automation levels vary by function—acquisition, analysis, decision selection, and implementation—rather than following one global scale. Human performance consequences should guide design choices within each function's acceptable range.
A meta-analysis of 18 experiments found automation improves routine performance but degrades failure recovery and situational awareness—most sharply when automation moves from analyzing information to selecting actions. Operators monitoring automated decision-makers lose the skill and awareness needed to take over.
Elish demonstrates that when automated systems malfunction, investigators and media assign responsibility to nearby human operators regardless of their actual authority or information access, protecting the technology's reputation while the operator absorbs moral and legal liability.
MGI classified US occupations as people-centric, agent-centric, robot-centric, or hybrid based on automatable task share. People-centric roles (one-third of jobs, ~$71k) and agent-centric roles (40% of jobs, ~$70k) pay nearly equally despite vastly different automation exposure.
When job tasks multiply together rather than add (O-ring production), automating some tasks concentrates worker time on what remains, raising its quality and value. This means standard exposure indices underestimate worker resilience by assuming static task value.
Show all 12 sources
Research identifies two feedback failures: automating entry-level work destroys the apprenticeship ground needed to build expert verifiers, and experts' own labels digitize their tacit judgment into replacements for themselves, narrowing future learning opportunities.
Evaluation of agentic systems shifts evidence from final responses to full interaction sequences, and scoring procedure from correctness alone to process quality, recoverability, coordination, and robustness. This pattern appears across multiple agent benchmarks as a coherent design move.
Single task-success metrics obscure how agents achieve results across memory, context, and verification layers. Research shows identical success rates can mask enormous differences in efficiency, reliability, and deployment readiness—requiring harness-level benchmarks that measure trajectory, memory hygiene, and verification costs.
Automated benchmarks both overstate and understate capability by privileging precisely-specified, auto-gradable tasks. Open-world evaluations of long-horizon messy tasks through qualitative log analysis—with cost explicitly reported—correct these distortions and catch emerging capabilities earlier.
BenchShield enables benchmark operators to issue claims about valid task completion grounded in recorded infrastructure evidence rather than terminal scores alone. This shifts from a single number to a verifiable claim about whether an agent followed the intended evaluation path.
Kwa's 8-parameter model predicts over 99% automation of AI R&D by mid-2032, matching the complex AI Futures Model by replacing poorly-defined assumptions with direct capability metrics and simpler production functions.
Partial instruments exist for individual conditions in isolated settings, but none measures the full socio-technical system the paper identifies as necessary. Visibility has a model-side measure (chain-of-thought disclosure), containment has incident-level counts, and recoverability has rollback timing, yet none bridges all four or captures human-institution factors.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent Evaluation
- Human Performance Consequences of Stages and Levels of Automation: An Integrated Meta-Analysis
- A Model for Types and Levels of Human Interaction with Automation
- PostTrainBench: Can LLM Agents Automate LLM Post-Training?
- Keeping an Eye on AI: A Framework for Effective Human Oversight of AI Systems
- Some Simple Economics of AGI
- Towards a Science of AI Agent Reliability
- BenchShield: Formal Model-Backed Instrumentation for Reward Integrity in LLM-Agent Evaluation Infrastructure