Can AI agents reliably complete healthcare administrative workflows?

HEALTHADMINBENCH is a benchmark of four deterministic GUI environments — an electronic health record (EHR) viewer, two payer authorization portals, and a fax submission portal — built to test LLM-based computer-use agents (CUAs, agents that observe screenshots or interface elements and act via clicks and typing) on 135 expert-defined administrative tasks across three workflow types: Prior Authorization, Appeals and Denials Management, and Durable Medical Equipment (DME) Order Processing. Each task is decomposed into 1,698 verifiable subtasks, checked by 1,177 deterministic and 521 LLM-based verifiers. Evaluating seven agent configurations built on five frontier models, the authors report that "the strongest agent achieves a 36.3% full-task success rate," held by Claude Opus 4.6 CUA, while "GPT-5.4 CUA attains the highest subtask success rate (82.8%)."

The paper attributes the gap between high subtask success and low full-task success to document handling and cross-system data transfer: administrative work requires "multi-step interactions across legacy enterprise systems that lack API support," and the environments were modeled on "over 100 hours of observation of real administrative tasks, during which staff routinely transfer information across multiple systems." Its discussion states plainly that agents "can often execute individual actions correctly, yet still struggle to reliably complete full administrative workflows, particularly when tasks require document handling, cross-portal coordination, and sustained state tracking over many steps."

This sharpens, in a single healthcare-specific domain, the same divergence Why do AI agents fail at workplace social interaction? found across general office work, and offers a concrete data point for Does a single benchmark score actually predict agent readiness?: subtask execution and end-to-end completion behave as separate, only loosely correlated axes, so a leaderboard built on either alone would rank agents differently. It also stands in tension with Can API-first agents outperform UI-based agent interaction?: HealthAdminBench's environments are explicitly built to simulate systems that lack API access, which is precisely the GUI-bound, cross-system condition AXIS argues produces slower, less accurate agent behavior than API-first interaction.

The benchmark's environments are synthetic and deterministic, and the paper's own limitations section notes it is "static and retrospective," abstracting away CAPTCHAs, multi-factor authentication, session timeouts, and the interface drift and policy heterogeneity of real hospital deployments — so the 36.3% figure describes a controlled proxy, not production payer portals, and likely overstates what agents would achieve against live systems with weaker instrumentation. What the excerpt does establish is that current frontier CUAs are far from reliable on full administrative workflows even under favorable, simplified conditions, which argues for continued human oversight before deployment rather than near-term automation of revenue-cycle work.