Chatbot leaderboards rank models by popularity — but popularity isn't the same thing as actually being good or correct.

This explores whether the numbers we use to rank chatbots tell you how good a model actually is. The corpus gives a split answer. Benchmarks reliably measure something, but that something is often narrower than "quality": a model's performance on short exchanges, inside a particular software setup, as judged by people who respond to style. The best-known positive case is Chatbot Arena. Its 240,000+ crowdsourced head-to-head votes produce rankings that agree with expert raters, mainly because the questions people submit are varied enough to separate strong models from weak ones Can crowdsourced votes reliably rank language models?. So human preference at scale is a real signal, not noise.

The catch is what human preference responds to. In imitation training, smaller models learned to copy ChatGPT's confident, fluent style. Human evaluators rated them as close to ChatGPT, yet they gained nothing in factual accuracy or in handling new tasks Can imitating ChatGPT fool evaluators into thinking models improved?. A separate line of work explains why this happens. Chatbots use language that signals expertise, and users' trust follows the tone of the answer rather than its accuracy Does chatbot language style actually shape how much we trust it?. Put these together and the risk is clear: any evaluation that relies on people saying which answer they like can be partly gamed by sounding authoritative.

A second gap is the shape of the test. Most benchmarks hand the model one complete instruction. When the same information arrives gradually over a natural conversation, accuracy falls from about 90% to 65%. Models lock onto an early guess and can't recover, a habit the research ties to RLHF rewarding helpfulness over asking clarifying questions Why do AI assistants get worse at longer conversations?. Alignment scores have a similar blind spot. A model can be honest and harmless and still handle conversation badly: it loses track of shared context and ignores basic conversational norms, and safety-focused evaluations don't capture this Can ethically aligned AI systems still communicate poorly?.

The most surprising finding may be that a benchmark score is not purely a property of the model. On Terminal-Bench, improving the execution setup around a model (the "harness" that runs its commands and manages its steps) raised scores for several models without changing their weights at all. GPT-5.6 reached 95.3% this way, and DeepSeek-V4 Flash gained 5.4 points Can execution harnesses lift model performance without retuning weights?. When two leaderboard entries differ, part of the gap may come from engineering rather than intelligence. Accuracy-only rankings also leave out cost and speed across a whole multi-step task. A 35B-parameter model trained on execution-grounded data competes with much larger ones at a fraction of the cost, which changes which model is "best" for real work Does model efficiency matter more than peak capability for real work?.

The practical lesson: treat a leaderboard number as a measurement of a model, in a particular harness, on single-turn tasks, judged partly on style. Before trusting it for your own use, ask about multi-turn behavior, factual accuracy, and cost. One caveat: this collection doesn't directly cover benchmark contamination (test questions leaking into training data) or benchmark saturation, so those parts of the question go unanswered here.