Why do GUI agents fail when leaving the lab?

The Qwen-UI-Agent report opens from a diagnosis: GUI agents "remain largely optimized for simulated benchmarks, leaving a substantial gap between benchmark performance and real-world utility." The promise it wants to reach is a "general purpose executor over existing application ecosystems" that completes tasks "without requiring every service to expose a dedicated API." The conclusion turns the diagnosis into a design claim: results "support a systems view of GUI agent development where models, environments, data flywheel, training, and harness are co-designed around real-world usage."

The abstract lists what co-design means in practice. On the environment side, diverse sandboxes are paired with a large-scale real-device mobile runtime. On the action side, a unified action space interleaves GUI operations with CLI execution and emits batched actions in a single model turn. On the data side, an "AutoResearch-style" flywheel uses agents to construct tasks and environments, diagnose failures and plan the next iteration. On the training side, SFT, action RL and online RL are combined, with online RL run on trajectories over 100 turns across more than 10,000 concurrent environments. A "lightweight harness layer" then handles proactive service initiation and stateful workflows across mobile and computer. The reported scores are 82.1% on MobileWorld, 92.2% on MobileWorld-Real, 97.5% on AndroidDaily, 79.5% on OSWorld-Verified and 40.0% on OSWorld-v2.

This is a builder's version of the worry in Does a single benchmark score actually predict agent readiness?: that benchmark success and deployment readiness come apart. Here the response is to change the training environment and the surrounding system, not only the evaluation. The premise also sits against Can API-first agents outperform UI-based agent interaction?. AXIS prefers API calls over sequential UI steps, while this report starts from the GUI because services often expose no API. Its action space adds CLI execution alongside GUI operations, so the position looks less like GUI-only and more like a hybrid. The gap it names is also visible in Why do AI agents fail at workplace social interaction?, where complex UI navigation is one of the hardest failure modes.

The excerpt does not show that co-design is what produced the scores. It gives no baselines, ablations or per-component contributions. It does not say how MobileWorld-Real differs from MobileWorld, or why OSWorld-v2 (40.0%) sits so far below OSWorld-Verified (79.5%). Nothing in it evaluates the proactive service initiation the abstract emphasizes. Its "real-world" evidence is performance on named benchmarks, including real-device ones, not measurements from deployment. It is also silent on whether planning and grounding are separated inside the single model. The systems view is best read as the paper's design thesis, and the benchmark-to-utility gap it opens with is still not measured directly.