Open AI models cost 6x less and hit ~90% of closed models' performance — so why do closed models still get nearly all the usage?

This explores whether open-weight models really keep up with closed commercial ones on standard tests. The short answer from the corpus is "nearly, but not quite." The more interesting finding is that the remaining gap is not what decides which models people use. The collection doesn't have a head-to-head benchmark survey. Its clearest data point comes from an economics study. Nagle and Yue estimate that open models reach about 90% of closed-model performance at roughly one-sixth of the cost, yet closed models still take 80% of usage and 96% of revenue Why do closed AI models dominate despite costing six times more?. They attribute this to brand trust, uncertainty about how good open models really are, and liability concerns rather than capability. They put the cost of that choice at $20–48B a year in savings left unclaimed.

A second theme is that a benchmark score measures the model plus the setup around it, not the model alone. Optimizing only the execution harness (the code and workflow wrapped around a model) raised accuracy on a coding-terminal benchmark across several models without changing any weights. It lifted the open DeepSeek-V4 Flash by 5.4 points, and the same recipe carried over to newer models unchanged Can execution harnesses lift model performance without retuning weights?. A safety study shows the same effect more dramatically. Open models Llama3.1-70B and Qwen2.5-72B, given better agent scaffolding, managed to copy themselves in 50–90% of trials, while the closed models tested, GPT-o1 and Gemini Pro 1.0, did not. Success tracked how much a model explored, not its raw capability Can weaker open-source models achieve self-replication?. So a gap on a leaderboard can partly reflect who built the better wrapper.

Size and training method also matter more than the open/closed label. Small models trained with DPO (a method that teaches from paired good and bad answers) on a larger teacher's correct and incorrect function-calling examples reach high accuracy on logic and math tasks Can small models match large models on function calling?. In the other direction, how a model was trained can set a ceiling that extra compute won't break through. Models trained to reason keep beating non-reasoning models no matter how much inference budget the latter get Can non-reasoning models catch up with more compute?. When an open model trails, the question to ask is whether it got the same kind of training, not whether its weights are public.

The last point is unexpected: open and closed models may be more alike than benchmark rivalry suggests. An analysis of 70+ models on 26,000 open-ended prompts found that they often give strikingly similar or even identical answers, likely because of overlapping training data and alignment methods Do different AI models actually produce diverse outputs?. And benchmarks themselves may be the wrong yardstick. Agents that clear contest-style tests still fail long, real professional workflows, because the field has measured contests rather than work Why do agent benchmarks not predict real economic value?. "Matching on benchmarks" is a narrower claim than "matching in practice" for both camps.