A tool that predicts when you'll want to step in on a web-browsing AI — could it also work for coding or desktop agents?
Can intervention prediction from web tasks generalize to other agent domains?
This explores whether a model trained to predict when a person wants to step in and take over from a web-browsing agent could do the same job for agents working elsewhere, such as coding, desktop software, or multi-agent systems.
This explores whether a model trained to guess when a user wants to step in and correct a web agent could carry that skill over to agents working in other settings. The short answer is that the corpus doesn't test this directly. The one study on intervention prediction stays inside web tasks. There, fine-tuned models that predicted interruption moments were 61–63% more accurate than baselines, and users rated the agents 36.8% more useful in a live study Can models predict when users want to intervene in web agents?. That shows the idea works on the web. It doesn't show how far it travels.
The corpus does say something about what makes agent skills transfer, which helps you make an informed guess. One pattern keeps coming up: skills learned at a finer grain and in more abstract form travel better. Agent Workflow Memory pulls out small reusable routines and strips away example-specific details. Its gains actually got *larger* as the gap between training and test tasks grew Can agents learn reusable sub-task routines from past experience?. Language world models trained on very large numbers of agent trajectories transferred across seven domains Can language models learn to simulate agent environments?. AIDE2's gains held up on a weather-forecasting benchmark well outside what it was tuned on Do AIDE2's improvements transfer to unseen tasks?. The lesson for intervention prediction: if a model learns general warning signs (the agent is about to do something irreversible, is drifting from what was asked, or is stuck in a loop), those signs could plausibly hold in other domains. If it learns web-specific cues, like a particular checkout button, they probably won't.
There is also a reason for caution. Agents trained on fixed expert examples are limited to what the people who built the dataset thought to include Can agents learn beyond what their training data shows?. Intervention data is that kind of curated human record: it captures where *these* users, on *these* sites, chose to step in. The moments when people want control in a coding agent or a multi-agent pipeline may look nothing like that. Multi-agent systems are a sharp example. When a harmful goal is split across subtasks that each look harmless, the moment someone should step in may not be visible at any single step Can task decomposition hide harmful intent across agents?. A predictor that watches one action at a time would miss it.
A less obvious point from the corpus: predicting intervention may matter less as a model skill than as part of the surrounding infrastructure. Chan and colleagues argue that agents need outside systems for tracing actions back to agents, shaping how they interact, and detecting and fixing harm What infrastructure must exist beyond training agents?. A tool that predicts interventions fits naturally into that third role. Interface design is a related lever: changing interfaces so agents perceive and act more reliably improved task completion without hurting human usability Can interface design help agents complete computer tasks better?. Both suggest that when people step in may depend as much on how the environment is built as on what the agent learned. So rather than asking whether one predictor transfers, the more useful question may be which parts of the 'when should a human step in' signal are general, and which depend on the setting the agent works in.
Sources 8 notes
Fine-tuned models that predict user intervention moments and integrate those predictions into live web agents achieved a 36.8% improvement in user-rated usefulness during a real-world task study, with 61.4–63.4% better intervention prediction accuracy than baselines.
Agent Workflow Memory induces sub-task routines at finer granularity than full tasks, abstracts example-specific values, and compounds them hierarchically. This produces 24.6% relative gain on Mind2Web and 51.1% on WebArena, with larger gains as train-test gaps widen.
Qwen-AgentWorld demonstrates that native language world models trained via next-state prediction on 10M+ trajectories outperform real-environment training on three benchmarks and transfer across seven domains, positioning next-state prediction as a foundation objective for agents.
The paper reports that AIDE2's improvements transfer to four held-out benchmarks spanning machine learning, algorithm engineering, and physics-based weather forecasting—the last being outside the selection task distribution. This demonstrates transferable gains beyond overfitting to the selection set.
Agents trained on static expert datasets cannot learn from their own failures or generalize beyond demonstrated scenarios because they never interact with environments during training. Competence is capped by what curators imagined, not by agent capacity.
Show all 8 sources
SafeFlow demonstrates that multi-agent systems' core strength—splitting tasks and specializing roles—creates a safety blind spot where malicious intent can be distributed across steps that each appear benign individually, with harm emerging only in composition.
Chan et al. argue that external technical systems modeled on internet protocols are essential beyond behavior training. These systems must handle three core functions: attributing actions to agents, shaping their interactions, and detecting and remedying harm.
UI-Verse benchmark testing found that augmented heuristics targeting agent perception, grounding, and control consistently improved agent task completion and efficiency. Human studies confirmed these designs preserved usability for human users with no observable regressions.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Agent Learning via Early Experience
- Nex-N1: Agentic Models Trained via a Unified Ecosystem for Large-Scale Environment Construction
- Evaluating Language-Model Agents on Realistic Autonomous Tasks
- A Framework for Studying AI Agent Behavior: Evidence from Consumer Choice Experiments
- Demystifying Agent Skills: Why They Work-Until They Don't
- Design Principles for Human-Agent Interaction
- Unaccountable Delegation, Fading Skills: Mapping the Risks of Workplace AI Agents
- Qwen-AgentWorld: Language World Models for General Agents