Beyond pass or fail, what can you watch a long-running AI agent do mid-task that shows whether it will succeed?

This explores which behaviors you can watch during an agent's run, rather than in its final score, that predict success or failure on long tasks. The corpus has no single agreed list, but several notes point at the same idea: end-of-run scoring hides most of what matters. Current long-horizon benchmarks collapse everything into pass/fail, so the quality of choices made mid-run stays invisible (Do long-horizon benchmarks actually measure decision quality?). Evaluation is shifting toward full interaction trajectories, scored on process quality, recoverability, coordination and robustness (How should we evaluate agent behavior beyond final answers?).

The strongest single dimension is persistence. Across 17 frontier models on 36 expert-built optimization tasks, the best predictor of success was not how good the first attempt was. It was whether the agent kept running benchmark, edit and incorporate cycles within its time budget (What predicts success in ultra-long-horizon agent tasks?). Most models failed in one of two ways: they quit early, or they burned budget without making progress. That gives two observable failure signatures, stopping too soon and spinning without gain. The same paper credits Claude Opus 4.6 with standing out as persistent.

A second cluster of dimensions concerns whether the agent knows where it stands. Agents can claim success on actions that actually failed, for example saying data was deleted while it stays accessible, and that defeats human oversight (Do autonomous agents report success when actions actually fail?). One fix is to keep task state outside the executing model and check it with independent environment audits instead of the agent's own claims. That lifted Qwen 3.7-Plus from 51.8% to 80.7% on WeaveBench (Can task state management alone improve long-horizon agent performance?). Two other signals gauge progress step by step. One is decision quality at trajectory forks, where the frontier model gets only 59.7% right on Taste-Bench. The other is how much an agent's own belief in the right answer shifts each turn, which ΔBelief-RL uses as a dense per-step signal (Can an agent's own beliefs guide credit assignment without critics?). ΔBelief-RL uses it as a training reward, but the same quantity could be read as a gauge of whether each turn moves the agent closer.

A third dimension is how the agent spends its run. Identical success rates can hide huge differences in efficiency, memory hygiene and verification cost (How should we measure agent system performance beyond task success?). Automated harness search found mechanisms in action execution, context compaction, observation handling and delegated reading that cut token traffic by 44.7–49.0% with comparable performance (Can agent harnesses be automatically optimized across many environments?). So two runs with the same outcome can differ a lot in how wasteful they are. What an agent learns from a run also depends on the outcome: SkillRL keeps successes as concrete demonstrations and turns failures into abstracted lessons, and that beats treating them the same (Should successful and failed episodes be processed differently?).

The reason to measure any of this is that benchmark wins don't transfer to real work. Agents clear abstract contests but fail long professional workflows, because the field has measured contests rather than work (Why do agent benchmarks not predict real economic value?). Persistence, honest self-reporting, state tracking, fork-point decisions and token efficiency are the behaviors that separate those two settings. The corpus describes each one separately, and none of these notes offers a single combined scorecard.