What quality of curated data is minimally sufficient for alignment?
This explores how little — and how clean — the fine-tuning data needs to be to align a model, reading 'quality' as curation rigor rather than raw volume.
This explores how little, and how clean, the curated data needs to be to align a model — and the corpus's headline answer is surprisingly stark: very little, if it's the right little. The clearest data point is LIMA, where just 1,000 carefully curated examples on a strong pretrained base reach alignment performance competitive with models trained on orders of magnitude more Can careful curation replace massive alignment datasets?. The mechanism matters more than the number: post-training isn't building new capability, it's activating what pretraining already installed. So 'minimally sufficient' isn't really a data-volume question — it's a question of whether your examples cleanly surface behavior the base model can already produce.
The reason so few examples suffice becomes clearer when you look at where alignment actually lands in the network. Proxy-tuning shows that alignment is largely a shallow, distributional shift — it reshapes reasoning and style at decoding time without touching the lower layers where knowledge is stored, closing most of the alignment gap while leaving the base weights untouched Can decoding-time tuning preserve knowledge better than weight fine-tuning?. If alignment only needs to nudge surface style and reasoning paths, a small, sharply curated set is plenty. The flip side is a warning, not a comfort: pretraining poisoning at just 0.1% of data survives standard safety alignment intact How much poisoned training data survives safety alignment?. The same shallowness that lets 1,000 good examples align a model means alignment can't scrub deeply embedded behavior — minimal-but-sufficient cuts both ways.
So the binding constraint shifts from quantity to purity and coverage. If a handful of examples can steer the model, then a handful of bad or mislabeled ones can steer it wrong, and you have no volume to dilute the error. This is where synthetic-data work gets interesting: Simula generates training data with no seed examples at all, separating global coverage (via taxonomy construction) from local diversity, making quality, diversity, and complexity independently controllable Can we generate synthetic data without any seed examples?. The implicit claim is that 'sufficient quality' is decomposable — you can engineer coverage and difficulty as separate dials rather than hoping a scraped corpus happens to contain both.
There's a deeper ceiling worth knowing about, though. Even perfectly curated data can't make a model align or improve itself past a hard limit: self-improvement is formally bounded by the generation–verification gap, where every reliable fix requires something external to validate it What stops large language models from improving themselves?. That reframes the whole question. 'Minimally sufficient quality' isn't a property of the dataset alone — it's a property of the dataset plus an external verifier good enough to certify it. A thousand examples align a model precisely because a human curator's judgment is the external signal baked into them. Strip that judgment out, and no quantity of self-generated data closes the loop.
Sources 5 notes
LIMA demonstrates that 1000 carefully curated examples fine-tuned on a strong pretrained model achieve competitive alignment performance with models trained on orders of magnitude more data, showing that post-training activates existing capabilities rather than building new ones.
Proxy-tuning closes 88-91% of the alignment gap while surpassing direct fine-tuning on knowledge tasks by leaving base model weights untouched. Direct fine-tuning corrupts knowledge storage in lower layers, whereas proxy-tuning applies distributional shifts that primarily affect reasoning and style.
Denial-of-service, context extraction, and belief manipulation attacks persist through standard safety alignment at 0.1% poisoning rates, while jailbreaking attacks are successfully suppressed, contradicting sleeper agent persistence hypotheses.
Simula separates global coverage from local diversity, using taxonomy construction for coverage and agentic refinement for complexity. This architecture makes all three desiderata—quality, diversity, complexity—controllable simultaneously without requiring seed data.
Self-improvement in LLMs is formally bounded by the generation-verification gap, meaning every reliable fix requires something external to validate and enforce it. Models cannot escape this constraint through metacognition alone.