Do base models find more solutions than post-trained ones?

The paper extends the RL "sharpening hypothesis" — that post-training amplifies a few high-reward behaviors already present in the base model, trading solution coverage (pass@K) for single-shot accuracy (pass@1) — from math and coding into agentic tool-use tasks, a domain where the authors expected this trade-off not to hold because "multi-turn tool use and interaction may require capabilities newly acquired during post-training." Testing 14 base/post-trained checkpoint pairs across four model families (Gemma-4, Ministral-3, Qwen2.5, Qwen3.5) on three agentic benchmarks (BFCL v4 multi-turn, WebShop, ACEBench — 42 cases total), they find base models equipped with only "a simple system prompt with relaxed tool-calling and parsing interfaces," no fine-tuning, "catch up to their post-trained counterparts as the rollout budget K grows" and eventually surpass them: on WebShop, a harnessed gemma-4-31B base model reaches "over 85% pass@128... vs. 56% for RL."

The mechanism they give is bimodalization: post-training does not uniformly raise or lower per-task success probability, it pushes the distribution toward two extremes — "always solved or never solved" — which raises pass@1 and makes repeated rollouts on the same prompt agree with each other (high "pass^K" consistency), but forecloses the long tail of rare-but-reachable solutions the base model can still find at large K. They formalize the resulting gap as "Sharpening Tax," Tax_X(K) = X_Base(K) − X_Post(K), a scalar estimable from a handful of rollouts and reported as "prevalent in most settings" across the 42 model-benchmark pairs. A second finding qualifies the first: the rollout budget k* at which base models cross over post-trained ones shrinks as model scale grows, so whether sharpening helps or hurts must be judged jointly with scale and test-time budget. To offset the tax during training, they propose posterior-tempered group sampling (PTGS), which adapts sampling temperature to per-prompt difficulty and "pays a smaller tax than the fixed-temperature baseline."

This extends, rather than repeats, Do base models already contain hidden reasoning ability? and Does RL teach reasoning or just when to use it?: both describe post-training as eliciting or scheduling capability the base model already holds rather than installing new capability, and this paper supplies the first coverage-based evidence for that claim outside math and coding — in multi-turn tool use, where the authors themselves expected the opposite going in. It sits in tension with Can reinforcement learning discover reasoning strategies base models cannot?, which reports RL-trained models beating base models at all pass@k levels; the difference is training regime — that result comes from prolonged RL with explicit entropy control and reference-policy resetting against collapse, while Sharpening Tax is measured on standard open-source post-trained checkpoints whose training recipes are undisclosed, so the two findings may mark different points on the same entropy-collapse spectrum rather than a contradiction. The bimodalization mechanism also gives a concrete behavioral signature to the collapse named in Does policy entropy collapse limit reasoning performance in RL?: rather than entropy declining smoothly, per-task success rates polarize toward 0 or 1.

What the excerpt does not establish is whether the tax is intrinsic to RL post-training itself or an artifact of the specific open-source checkpoints studied — the authors note their analysis is "observational rather than interventional" since the exact pre-training and post-training data and recipes are undisclosed. It also leaves open whether frontier-lab pipelines already using diversity-preserving techniques (of the kind PTGS or prolonged RL represent) pay the same tax as these academic checkpoint pairs. The stated implication, at the strength the evidence allows, is a reporting norm rather than a universal law: "post-training should be judged not only by its pass@1 but also by how much of the base model's potential it keeps," i.e., coverage metrics like Sharpening Tax should accompany accuracy whenever a lab claims post-training "unlocks" new agentic capability.