Does pretraining loss predict performance after reinforcement learning?

The paper argues that RL post-training has been "studied in isolation from the pretraining that precedes it," and that two basic questions stay open as a result: how pretraining choices shape the returns to RL compute, and what RL does to the model. Using chess as a controlled testbed, it pretrains language models from 5M to 1B parameters on human games, fine-tunes them on synthetic reasoning traces, and runs RL on chess puzzles with verifiable rewards. The headline result is a joint scaling law: "the post-RL performance at given RL compute level is well-predicted from the pretraining loss, and slope of the RL reward curves improves approximately linearly with the pretraining tokens." The discussion words the second half more cautiously, saying pretraining data scale is "closely associated with the slope of RL improvement."

The reasoning for the design is that standard LLM pretraining corpora are "vast and uncontrolled," so behaviors are hard to attribute to pretraining versus RL, and compute sweeps across both stages are "prohibitively expensive." A small domain with a full pipeline makes the sweep affordable. The two-part structure of the law gives a division of labor: pretraining loss sets the level RL-trained performance reaches, while pretraining data sets how quickly RL climbs. From this the paper derives a compute-allocation frontier. As total budget grows, "the optimal pretraining fraction tends to decrease," so RL should take an increasingly large share. Starting RL too early from weakly pretrained checkpoints "gives limited gains in our setting," which the authors read as RL remaining "initialization-dependent." The introduction frames this as a choice between two views on where to spend compute, the pretrained prior versus experience, and the result is a tradeoff with a frontier, not a win for either.

This extends When does RL actually extend reasoning beyond pretraining?, which states the pretraining-dependence of RL qualitatively on synthetic reasoning tasks. Here the same dependence shows up as a fitted relationship in a different controlled domain, and the "sufficient pretraining exposure" requirement echoes the headroom condition. It fits Does the choice of RL algorithm actually matter for reasoning? in making the pretrained model the main predictor of RL outcomes, but the practical lesson differs. That note concludes that effort belongs in pretraining. This paper says the optimal RL share of compute should grow as budgets grow. The two are not strictly contradictory, since one concerns which optimizer to use and the other how to split a budget, but they pull in different directions on where marginal compute goes. It also sits beside Can reinforcement learning discover reasoning strategies base models cannot? as another controlled attempt to say what RL adds beyond the base model.

The excerpt does not give the functional form of the law, the quality of the fit, the number of runs, or the RL compute range. Its final sentence, "the behavior also differs between pass@1 and pass@16," is left without content, so nothing here says whether the law holds equally for single-sample and multi-sample performance. The second question the paper poses, what RL does to the inherited policy, is announced but not answered in the passages available. Chess with human-game pretraining and verifiable puzzles is also far from natural-language LLM training, and the excerpt does not address whether the linear slope relation or the shrinking optimal pretraining fraction carries over. The defensible reading is an existence result: in one controlled setting, pretraining and RL returns are jointly predictable. It is not yet a rule for splitting budgets in production models.