Is the gap between open-weight AI models and the best closed ones shrinking, or is it a fixed head start that never closes?

This explores whether the gap between open models and the frontier is a fixed lag, or whether it changes depending on which open models you count and how you measure them. The short answer from the corpus: the 6-to-12-month figure is real, but it describes one specific slice of open models, and the other measurements point in different directions.

The 6-to-12-month number comes from looking at open models small enough to run on a single high-end consumer GPU, roughly 28B to 40B parameters. These match what frontier models could do 6 to 12 months earlier How quickly do open models catch up to frontier AI?. The same research group finds a much shorter gap of about four months when it compares the best open-weight models of any size with the best closed ones How far behind closed models are open-weight models?. So 'the lag' is really two numbers. One is a hardware-limited lag for models you can run at home. The other is a lag for the strongest open releases, which need data-center hardware to run.

The two measurements also disagree about which way things are moving. On LM Arena, where people vote on which model gives better answers, the consumer-GPU-sized open models are gaining Elo about 56% faster than frontier models (+125 vs. +80 per year). That points to the gap shrinking Are open models closing on frontier AI faster than expected?. But the capability-index comparison found the gap at the top widening, from about three months to about four. It also warns that the true gap may be larger than measured, because benchmarks are saturating and some proprietary models haven't been released yet How far behind closed models are open-weight models?. Both can be true at once. Small open models can be catching up on how good their answers feel to users while the open frontier falls slightly further behind on hard capability tests.

One more factor makes the lag less stable than it looks. About half of measured frontier progress on a hard science benchmark tracks increased inference spending, meaning models spending more compute at answer time, rather than gains that come without more compute. Frontier running costs are rising 3x to 18x per year Is benchmark progress outpacing the cost to achieve it?. A model squeezed onto one consumer GPU can't buy its way forward like that. So if frontier progress increasingly comes from spending more at answer time, the gap for at-home models could widen even while their efficiency keeps improving. Distillation (training small models on the outputs of big ones) and better GPUs, two of the forces keeping the lag steady How quickly do open models catch up to frontier AI?, push the other way.

The collection doesn't have long-run time-series data or forecasts, so it can't say whether the lag will hold for years. What it does show is that 'how stable is the lag' depends on three choices: which open models you count, whether you measure with benchmarks or human preference votes, and how much of frontier progress you treat as bought with compute.