How far behind closed models are open-weight models?

Epoch AI's Epoch Capability Index (ECI) — the organization's own aggregate benchmark score for model capability — finds that, since January 2026, the most capable open-weight models have lagged frontier closed models by an average of four months, with an average ECI gap of 8 points, "similar to the gap between GPT-5 and GPT-5.5." Epoch AI reports a 90% confidence interval of 7 to 11 ECI points for the gap. The finding updates the organization's own earlier estimate: an October 2025 Epoch AI analysis found open models lagging closed ones by an average of three months across January 2023 to October 2025, so the newer gap is "slightly larger."

Epoch AI calculates the time gap day by day over its January 1 to May 28, 2026 analysis window: for each day, it finds the open-weight model with the highest ECI score available, then asks what the most recent date was on which the closed state-of-the-art model was "not significantly better" than that open model. Because ECI scores carry estimation uncertainty, Epoch AI uses bootstrap resampling of its benchmark scores, treating an open model as having caught up if it beats the historical SOTA model in at least 5% of paired bootstrap samples — a significance threshold, not a point-estimate comparison. Under the stricter test, requiring the open model's point estimate to exceed the closed model's, the gap grows to six months, so the four-month figure is the more lenient of the two estimates Epoch AI reports.

Epoch AI flags two reasons the four-month estimate may understate the true gap: open models reportedly score worse on private benchmarks than on public ones, "plausibly because they more aggressively hillclimb on public benchmarks," and closed labs do not always release their most capable models. That caveat about benchmark-score reliability echoes the concern in Can frontier exams really measure cutting-edge AI capability?, where high scores on a closed-ended exam are explicitly said not to imply real autonomous-research ability — both notes treat a capability index as a proxy that can diverge from the ability it is meant to track. The finding also sits alongside How much of AI progress comes from better tools versus better weights?, which frames frontier progress generation-to-generation as uneven; Epoch AI's open/closed cadence gives that unevenness a specific, measured lag rather than a general description of "spiky" progress. It likewise shares a time-gap methodology with How fast is AI cyber autonomy actually advancing?, another case where a capability trend is measured carefully but its durability is left open.

The analysis covers only models with enough public benchmark coverage to be assigned an ECI score, so models labs withhold or benchmark differently fall outside the comparison, and the four-month figure describes an aggregate across many benchmarks rather than performance on any specific task or domain. Epoch AI's own framing — that the true gap is probably understated — suggests four months is closer to a floor than a central estimate of how far open models trail closed ones, and the excerpt says nothing about whether that gap will narrow or widen going forward beyond the one earlier comparison point.