Closed AI models often score higher, so why do so many users still run cheaper open ones?

This explores why many people choose cheaper open-weight models when closed frontier models score higher, and what that says about how people actually pick a model. The short answer from the corpus: 'performs better' is the wrong unit. Users rarely pick one model for everything. They pick per task, and on many tasks a cheaper model is good enough. The largest real-world evidence comes from OpenRouter's study of more than 100 trillion tokens. Open models like DeepSeek and Qwen handled over 30% of traffic, even in areas where closed models clearly led on capability How much do open-weight models actually get used in practice?.

The less obvious part is that cheapness isn't the whole explanation either. The same data shows price alone poorly predicts what people use. Requests spread across the whole price range instead of piling onto the cheapest option. Users trade cost against latency, reliability, reasoning quality, and how many kinds of task a model can handle Does LLM price alone determine which model users choose?. So open models aren't winning a race to the bottom. They're winning the many slots where their mix of trade-offs fits the job better than the top model's mix does.

Research on model routing shows why this can be a smart strategy rather than a compromise. FrugalGPT sends each query to a cheap model first and moves up to an expensive one only when the cheap model's confidence is low. That matched GPT-4's accuracy at up to 98% lower cost Can cheaper language models match GPT-4 accuracy?. Avengers-Pro goes further. It groups queries by topic and sends each group to the model that handles it best, which beat GPT-5-medium by 7% on accuracy or matched it at 27% lower cost. Earlier, ten small 7B models with routing outperformed GPT-4.1 Can routing beat building one better model?. Seen this way, a leaderboard average hides the fact that different models win on different queries. Picking the right model for each query can matter more than having the single best model.

There is a limit. Running many cheap models only catches up with a strong one when something can check the answers, such as unit tests, proofs, or type checks. Without a checker, sampling more answers gives you more chances to be right but no way to tell which answer is right When can weak models match strong model performance?. That helps explain where open models do well: coding and other tasks with clear right answers. It also explains where paying for a frontier model still makes sense: open-ended work where nobody can quickly confirm the output is correct.

One more twist cuts against the idea that the stronger model is always the safer choice. When models edit long documents, weaker models tend to fail visibly by deleting content. Frontier models tend to fail quietly, introducing subtle errors while the document still looks intact Does model capability change how documents degrade?. A cheaper model whose mistakes are easy to spot can be easier to use safely in a workflow than a stronger model whose mistakes look polished. The corpus doesn't directly cover other reasons people often give for choosing open models, such as data privacy, self-hosting, or freedom to fine-tune. Those are open questions here, not settled findings.