How fast are frontier AI agents improving at long tasks?

METR's time horizon study proposes a metric for comparing AI capability to human capability directly: the "X%-(task completion) time horizon," defined as "the time humans typically take to complete tasks that AI models can complete with X% success rate." Testing 12 frontier models released between 2019 and 2025 on a combined 170-task suite built from HCAST, RE-Bench, and a new 66-task Software Atomic Actions (SWAA) suite, the paper reports that "current frontier AI models such as o3 have a 50% time horizon of around 110 minutes," and that this horizon "has doubled approximately every seven months since 2019, though the trend may have accelerated since 2024." Extrapolating the trend, the authors write that "within 5 years, AI systems will be capable of automating many software tasks that currently take humans a month" — with an 80% confidence interval for a one-month (167-hour) time horizon spanning "mid-2028 to mid-2030," or "as soon as early 2027 if the 2024–2025 trend continues."

The paper attributes the growth in capability "primarily" to "greater reliability, ability to adapt to mistakes, logical reasoning, and capacity for tool use," rather than to any single architectural leap. Methodologically it borrows from human psychometrics (Item Response Theory): instead of scoring tasks individually, it fits a logistic regression across the task suite, comparing the geometric mean of successful human baseline times — drawn from "over 800 baselines totaling 2,529 hours" — against each model's binarized pass/fail outcomes, to estimate the model's horizon.

This trend line is a direct successor to When do AI agents outperform human research experts?, which the current study folds in as one of its three constituent task suites; where RE-Bench fixes a time budget and compares agent-to-human ratios at that budget, this paper inverts the framing to ask what task length a model can clear at a fixed success rate, turning budget-by-budget comparisons into a single exponential trend. The paper's own caveat that performance is weaker on "less structured, messier tasks" echoes Do automated benchmarks hide what frontier AI systems can really do?: both note that automatically-scored, well-specified benchmark tasks likely overstate real-world readiness. Its discussion of "dangerous capabilities" tied to increased autonomy also bears on Can current AI agents replicate and adapt autonomously?, since the same doubling trend this paper measures would, if it continues, shorten the time before autonomous-replication-style thresholds are reached.

The excerpt is explicit that its own trend may not generalize: "realistic tasks rarely have automatic scoring, and often require dealing with other agents, resource constraints, dynamic environments, and high reliability requirements," so the five-year, month-long-task extrapolation holds only "if these results generalize to real-world software tasks" — a conditional the paper states but does not test. What the paper establishes is a measured trend on a specific 170-task suite scored against specific human baseliners; the headline forecast is the authors' own extrapolation under stated uncertainty, not an observed real-world automation timeline.