If an AI can describe what it learned in fewer bits, does that mean it genuinely understands — or just memorized less?
Can compression length really indicate how well a model generalizes?
This explores whether the *size of a model's compressed description* — how few bits it needs to encode what it learned — is a trustworthy proxy for real generalization, or whether shorter codes can mislead.
This explores whether compression length is a trustworthy proxy for generalization, and the corpus makes a strong case that the two are deeply linked — with important caveats about *which* compression you measure. The starting premise is almost startling: language modeling is literally equivalent to lossless compression. A Chinchilla model trained only on text can out-compress FLAC on audio and PNG on images, not because it memorized those formats but because it adapts on the fly using its context window Can text-trained models compress images better than specialized tools?. Generalization, in this framing, isn't something separate from compression — it *is* compression working across a domain it never saw. That's the intuition behind treating shorter codes as a signal of learning, and it's reinforced by work deriving optimal training itself from a pure compression objective, where maximizing the data compression ratio improves the scaling-law exponents, not just the constants Does optimal language model learning maximize data compression?.
But 'compression length' is doing a lot of work, and the sharpest note in the corpus is a warning about naive measurement. Prequential coding — the classic 'how many bits to describe the data as you go' method — conflates genuine learned content with confounds like parameter count and raw data entropy. Requential coding fixes this by coding *only where a student disagrees with a teacher*, isolating what was actually learned and yielding codes orders of magnitude shorter than prequential ones, independent of model size Can we measure what a model truly learned?. So the answer to 'can compression length indicate generalization' hinges on whether you're measuring the model's *true simplicity* or just an artifact of scale. Two models with the same test accuracy can have wildly different honest description lengths.
There's also a phase-transition story that gives compression length a concrete mechanism. Models memorize until they hit a fixed capacity — roughly 3.6 bits per parameter — and only *then* does grokking kick in, flipping from storing examples to genuine generalization When do language models stop memorizing and start generalizing?. Before that threshold, a model can score well while its 'compression' is really just storage; the description length hasn't collapsed because nothing general has been found yet. This is exactly why raw code length alone can mislead — the interesting signal is the *drop* that accompanies the memorization-to-generalization transition, not the absolute number.
The corpus also hints at where compression and generalization diverge. LLMs compress concepts far more aggressively than humans do: they nail broad category structure but discard the fine-grained, context-dependent distinctions humans keep for situated action Do LLMs compress concepts more aggressively than humans do?. Shorter codes can mean better generalization — or they can mean you've thrown away nuance that mattered. And generalization has flavors compression length doesn't obviously capture: length generalization (10-digit to 100-digit addition) comes from transformers reusing shared attention-head machinery across related tasks Can length generalization transfer between different related tasks? and from iterative self-training on their own correct solutions Can transformers improve exponentially by learning from their own correct solutions?, which is about reusable computational structure more than description size.
So the honest answer: yes, compression length is one of the best-grounded proxies we have — generalization and compression are formally the same coin — but only if you measure the *learned* simplicity rather than the confounded total, and only if you remember that maximal compression isn't always the goal. The reader's takeaway that they didn't know they wanted: the reason 'shorter description = better model' feels almost too clean is that most metrics quietly bake in parameter count and entropy, and the whole game is subtracting those out to see what the model actually understood.
Sources 7 notes
Chinchilla models trained exclusively on text achieve better compression rates on images and audio than FLAC and PNG by using their context window to adapt as task-specific compressors. This demonstrates that generalization operates through compression, not specialization.
Research shows that optimal LM training can be derived from a lossless compression objective, yielding a Learning Law where all examples contribute equally in the optimal process. This approach improves scaling law coefficients, not just constants.
A teacher-student disagreement coding scheme produces compressed sizes independent of parameter count and data entropy, yielding orders of magnitude shorter codes than prequential methods. This isolates genuine learned content from confounding factors.
GPT-family models have a measurable memorization capacity of approximately 3.6 bits-per-parameter. When this capacity fills, a phase transition triggers grokking—the shift from memorization to genuine generalization. This capacity is a property of individual models, not training algorithms.
Using Rate-Distortion Theory on cognitive datasets, LLMs capture broad category structure but lose fine-grained distinctions humans preserve. LLMs maximize compression efficiency; humans trade compression for contextual meaning that enables situated action.
Show all 7 sources
Models trained jointly on related tasks reuse the same attention heads to handle length generalization, allowing shorter tasks to extrapolate beyond their training length. Pretrained models already contain this reusable computational scaffolding.
Standard transformers generalize from 10-digit to 100-digit addition by repeatedly generating solutions, filtering for correctness, and retraining—showing exponential (not linear) out-of-distribution improvement across rounds without saturation.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- From Tokens to Thoughts: How LLMs and Humans Trade Compression for Meaning
- Language Modeling is Compression
- How much do language models memorize?
- Requential Coding: Pushing the Limits of Model Compression with Self-Generated Training Data
- Extrapolation by Association: Length Generalization Transfer in Transformers
- Towards Optimal Learning of Language Models
- Self-Improving Transformers Overcome Easy-to-Hard and Length Generalization Challenges
- Computational structuralism: Toward a formal theory of meaning in the age of digital intelligence