Can models predict when users want to intervene in web agents?

Building on a taxonomy of four user collaboration styles (see Do users develop stable collaboration styles with web agents?), the paper trains language models to predict, at each step, whether a user is about to intervene, framing it as "a stepwise sequence prediction task" where the model estimates intervention likelihood "given the evolving task context." Across model backbones, these intervention-aware models "improve intervention prediction accuracy by 61.4–63.4% over baselines." The paper then deploys these models "into live web agents and evaluate them through a user study on real-world web tasks," finding "a 36.8% increase in user-rated agent usefulness" relative to baseline systems.

Mechanically, the model is fine-tuned via supervised finetuning to take "the history trajectory," "the current observation," and "a description of the agent-proposed action," and output one of two tokens, "<ask_user> or <agent_continue>," folding the intervention decision directly into the agent's action loop rather than handling it with a separate rule or heuristic. The paper tests two variants: "a general intervention-aware model" and "style-conditioned models adapted to specific collaboration styles," checking whether conditioning on the four collaboration styles sharpens prediction beyond one generic model for all users.

This pairs with, but measures something different from, the finding in Does oversight strategy prevent risky agent actions or just catch them? that interface strategy changed how often a problematic action occurred but not whether users could stop one once underway; here the lever is not restructuring who holds default control but predicting, from context, when a given user wants to be asked at all. It is one more entry in a design space that includes Can users steer AI agents better by revising plans mid-execution?'s per-step delegation — all approaches to the same underlying problem of deciding when to hand control back to the user, here solved by learning to predict the moment from a labeled corpus rather than by changing the interface structure itself.

The excerpt does not report the size or demographics of the user study behind the 36.8% usefulness figure, nor whether user-rated usefulness was measured on the same scale or task set as the offline prediction-accuracy gains, so the two figures should not be read as measuring the same thing. It also does not say whether the deployed models were the general or the style-conditioned variant. The implication the paper draws, and the excerpt supports only at this scope, is that "structured modeling of human intervention leads to more adaptive, collaborative agents" — a claim about this corpus and this deployment, not yet evidence that intervention-prediction training generalizes beyond web navigation tasks.