Should reliability requirements change with agent autonomy?

"Towards a Science of AI Agent Reliability" argues, as its fourth discussion recommendation, that "reliability requirements should scale with the degree of agent autonomy." It distinguishes two deployment modes: augmentation settings — "coding assistants, search copilots" — where "a human reviews and approves output before it takes effect, serving as a reliability backstop," and automation settings — "customer-service chatbots, unattended workflows" — where "the agent's output is the final action with no human buffer, so unreliability translates directly into failures."

The paper's example makes the stakes concrete: "an agent that succeeds on 90% of tasks but fails unpredictably on the rest may be a useful assistant yet an unacceptable autonomous system." The same accuracy and the same reliability profile can be adequate or disqualifying depending on where the human sits in the loop. The paper credits this backstop effect with explaining current adoption patterns: "this has enabled AI coding assistants to reach wide adoption despite imperfect reliability," because the human reviewer, not the model, is absorbing the cost of inconsistency, brittleness, and miscalibration.

This recommendation follows directly from Does AI capability growth translate to reliability gains? (sibling note): rather than treating reliability as a fixed bar every deployed agent must clear, it makes the bar context-dependent on autonomy level. It also sharpens the practical implication of Does a single benchmark score actually predict agent readiness?: deployment readiness is not a property of a model's benchmark score alone, but of the match between the model's measured reliability profile and how much of the decision loop the deployment hands to the agent unsupervised.

The paper states this as a discussion recommendation, not as something it tested empirically against its own 15-model, two-benchmark evaluation — it proposes no measured threshold for how much reliability a given autonomy level requires, and does not show that its twelve metrics predict failure rates differently in augmented versus unattended deployments. The implication the excerpt supports is a governance design principle — calibrate autonomy to measured reliability rather than to capability scores — not a validated finding about where that calibration line should sit.