Does AI capability growth translate to reliability gains?

"Towards a Science of AI Agent Reliability" argues that evaluating agents by mean task success rate "obscures critical behavioral properties," and proposes decomposing reliability into four dimensions borrowed from safety-critical engineering: consistency ("repeatable behavior"), robustness ("stability under perturbations"), predictability ("calibrated confidence"), and safety ("bounded severity when failures occur"). Across these four dimensions the paper defines twelve metrics "independent of raw accuracy" and evaluates 15 models on two benchmarks, τ-bench and GAIA. Its headline finding: "24 months of rapid capability gains have produced only small improvements in reliability" — models that are substantially more accurate "remain inconsistent across runs, brittle to prompt rephrasings, and often fail to understand when they are likely to succeed."

The decomposition is granular rather than a single score. Consistency splits into outcome consistency (does the agent succeed or fail the same way on repeated attempts), trajectory consistency (distributional and sequential similarity of the solution path across runs), and resource consistency (variance in cost and latency). Robustness splits into fault robustness (handling API timeouts or malformed responses), environment robustness (sensitivity to semantic-preserving changes like reordered JSON fields or renamed parameters), and prompt robustness (invariance to rephrased instructions). Safety combines compliance and harm severity using "the classical risk formulation of Kaplan & Garrick (1981)" — violation probability times expected severity — and is deliberately excluded from the paper's aggregate reliability score because "safety violations are inherently a tail phenomenon" that averaging would obscure. The paper grounds the exercise in real incidents it says accuracy scores missed: "Replit's AI assistant deleted a production database despite instructions forbidding it," "OpenAI's Operator made an unauthorized purchase that bypassed user confirmation," and "an NYC government chatbot provided illegal business advice" — in each case, "agents judged to be capable during internal assessments failed unreliably in deployment."

This sits alongside Does a single benchmark score actually predict agent readiness?, but makes an orthogonal rather than parallel claim: that note's axes decompose capability itself, while this paper holds capability fixed and shows reliability is a separate axis that does not rise with it — "a highly capable system can be unreliable... a less capable system can be highly reliable within its envelope." It also gives formal shape to the intuition in How should we measure agent system performance beyond task success? by naming trajectory consistency as a measurable sub-metric with its own distributional and sequential components, and by supplying a safety-critical-engineering pedigree (certification practice, explicit tail-risk quantification) that the trajectory-evaluation framing does not invoke. And it supplies a systematic counterpart to What shortcuts do high-accuracy agents actually take during benchmarks?: the catastrophic, accuracy-invisible actions that log analysis surfaces by hand are exactly what the harm-severity metric is built to quantify at scale rather than by inspection.

The result rests on two benchmarks (τ-bench, GAIA) and a single scaffold per benchmark, so it does not establish that the consistency and robustness gap generalizes across task types or agent architectures — the paper's own limitations section names "scaffold diversity" as untested. Safety scoring depends on LLM-based judging, which the authors say "introduces its own reliability concerns" into a metric meant to measure reliability, and temperature was fixed at zero throughout, which "may overestimate the reliability achievable when nonzero temperatures maximize accuracy." The claim the evidence actually supports is narrower than "agents are unreliable": it is that the labs whose models were tested have not yet optimized training for these four dimensions as targets distinct from benchmark accuracy — a claim about current training priorities, not a ceiling on what post-training could achieve.