Does optimal language model learning maximize data compression?

Most work on accelerating LM learning targets the model, optimizer, or data heuristically. This paper instead derives principles. It frames optimal learning through the "LM-training-as-lossless-compression" view: the objective is to maximize the data compression ratio. From that objective it derives a Learning Law — a property of the optimal-learning dynamics stating that, in the optimal process, all examples should be equally contributive to the model (validated on linear classification and real language modeling). And it shows empirically that optimal learning's payoff is concrete: it improves the coefficients of the scaling law, not merely the constant — meaning better learning shifts the whole compute-performance curve.

The keeper is the equivalence it leans on and the law it yields: if training is compression, then the best learning process is the one whose every example pulls its weight equally, and achieving that is what bends the scaling law favorably. It reframes "learn faster" from engineering tricks to a property of how contribution is distributed across data.

This deepens the vault's compression-as-learning thread. It extends Can text-trained models compress images better than specialized tools? from a property of trained models to a training objective, and it gives a theoretical complement to data-selection findings like Can we prune training data without hurting model performance? — though note the tension: data-pruning says examples differ in value, the Learning Law says the optimal process equalizes their contribution.