When a paper says its method 'equals or surpasses' a rival, which results are real wins and which are just ties?

This explores which benchmarks count as outright wins and which as ties when a paper says a method "equals or surpasses" a baseline. The corpus has no note that itemizes wins versus ties for one particular claim, and the question doesn't say which claim it means. So there is no scoreboard to give you. What the corpus does have is several worked cases of how "equals" and "surpasses" split apart, and they suggest what to check when you meet the phrase.

Start with the ties. Reasoning models with extended chain-of-thought show Do reasoning models actually beat standard models on optimization? on constraint-bound numerical tasks like optimal power flow. There the tie is the finding: extended thinking produces more text, not more iterative computation. A tie can also hide a win somewhere else. Does step-level confidence outperform global averaging for trace filtering? reaches accuracy gains comparable to naive majority voting, so accuracy is roughly equal. It does so with far fewer generated traces, so the win is in cost. When a claim says "equals," ask what the method saves instead.

Wins can also be worse than they look. Qwen2.5-Math-7B can reconstruct 54.6% of MATH-500 from partial prompts, yet it scores 0.0% on the newer LiveMathBench. That means some of the headline gains on familiar benchmarks are memorization, and Does RLVR success on math benchmarks reflect genuine reasoning improvement? shows that on clean benchmarks only correct rewards help. Does LLM math reasoning truly generalize or just pattern match? adds a second check. A model that "surpasses" on standard GSM8K-style problems can fall sharply when only the numbers change or an irrelevant clause is added. A benchmark win means less if the benchmark is old, public, or easy to pattern-match.

Whether you see a win or a tie also depends on the shape of the task. On problems that need accumulated intermediate steps, like graph connectivity, When does sequential reasoning beat parallel voting?. Under a matched token budget, though, Why does parallel reasoning outperform single chain thinking?, with up to 22% higher accuracy from majority voting. Both are real wins, on different kinds of problem, and an average across mixed benchmarks can blur the two into a tie. Even the scoring can manufacture ties: Can reading logit distributions break ties in LLM judging? shows that discrete judge outputs produce many tied comparisons, which continuous scores break.

The last piece is that the comparison itself is shaky. Do interactive evaluations actually solve the benchmark comparison problem? argues that comparability and reproducibility problems don't go away when you move to richer evaluations. If you tell me which paper or method makes the "equals-or-surpasses" claim, the useful next step is to check its wins against clean, post-release benchmarks and its ties against cost.