Can simply sorting training examples by difficulty — keeping some, dropping others — make AI training cheaper without hurting accuracy?
Can ranking examples by difficulty improve training efficiency within one dataset?
This explores whether sorting or filtering the examples inside a single training set by how hard they are (keeping some, dropping others, or changing their order) makes training cheaper or more effective.
This explores whether sorting the examples inside one dataset by difficulty can make training more efficient. The corpus says yes, with a catch: it depends which end of the difficulty scale you cut, and whether 'difficulty' is even the right thing to measure. The strongest evidence for yes comes from data pruning. When researchers scored each image by how hard the model found it (how often it got it wrong, how often it forgot it, how much it had to memorize it) and then removed the easiest ones, they could drop half of CIFAR-10 with no loss in accuracy Can we prune training data without hurting model performance?. The bigger result is that this beats the usual scaling law. Normally you need ever more data for ever smaller gains. Pruning the redundant easy examples can, in principle, make gains grow much faster as the dataset grows. Easy examples aren't harmful. They just stop teaching the model anything once it has learned them.
The other end of the scale is where things go wrong. In reinforcement learning with verifiable rewards (training a model on problems whose answers can be checked automatically), loading up on problems that are nearly impossible doesn't stretch the model. It damages it Do overly hard RLVR samples actually harm model capabilities?. When the model almost never succeeds, the rare accidental success gets treated as a huge win. That rewards whatever the model happened to do that time, such as repeating answers or skipping steps. These shortcuts then leak into skills the model already had. So ranking by difficulty does help, but the target is a middle band: drop what's too easy to teach anything and what's too hard to give an honest learning signal.
The less obvious finding is that 'difficulty' may be the wrong axis entirely. One curriculum study ordered fine-tuning data from *rare to common* instead of easy to hard, and it beat the standard easy-first approach Does ordering training data by rarity actually improve language models?. The argument is that for a pretrained LLM, what matters is less how conceptually hard an example is and more how far it sits from what the model saw during pretraining. Rare examples point to gaps in the model's knowledge. This changes what curriculum learning means: it becomes managing distance from what the model already knows, not building a learning path the way a teacher would. A related warning shows why ordering choices matter: RL training tends to lock onto one dominant format from pretraining within the first epoch and push the rest aside Does RL training collapse format diversity in pretrained models?. What the model sees early can shape where it ends up.
The same idea shows up outside of training, when choosing examples for a prompt. Picking few-shot examples for how much they reduce the model's uncertainty beats picking the ones most similar to the query Can optimal experimental design improve few-shot example selection?. Deliberately getting the model to fail on examples, then having it write down principles from those mistakes, improves reasoning without any new data Does learning from mistakes improve in-context learning?. Across all of these, the useful question isn't 'how hard is this example?' but 'how much does this example change what the model knows?' Difficulty is often a useful stand-in for that, but not always the best one.
Sources 6 notes
Research shows that ranking training examples by difficulty (EL2N, forgetting, memorization) and removing easy ones beats power-law scaling laws. On CIFAR-10, 50% of data was pruned without accuracy loss, and self-supervised metrics scaled the approach to ImageNet.
Training on nearly-impossible problems causes models to learn degenerate shortcuts rather than genuine reasoning, and these shortcuts contaminate pre-existing capabilities. Group-relative normalization treats rare accidental successes as high-advantage trajectories, reinforcing answer repetition and computation-skipping instead of sound reasoning patterns.
CTFT fine-tunes LLMs on rare data first because rarity signals distributional weakness, not conceptual difficulty. This reframes curriculum learning as managing distance from pre-training distribution rather than pedagogical scaffolding.
Controlled experiments show RL consistently amplifies one format distribution from pretraining within the first epoch while collapsing alternatives. The winning format depends on model scale, not necessarily performance, and is largely hidden when starting from proprietary pretrained models.
AIPD frames demonstration selection as budgeted active learning, choosing examples that maximally reduce test-set uncertainty. Two algorithms (GO and SAL) outperformed similarity-based methods across small, medium, and large language models.
Show all 6 sources
LEAP demonstrates that models achieve better performance on reasoning and math tasks by intentionally erring on few-shot examples, reflecting on mistakes, and deriving explicit task-specific principles—without additional labeled data or fine-tuning.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Echo Chamber: RL Post-training Amplifies Behaviors Learned in Pretraining
- In-Context Principle Learning from Mistakes
- Mechanistically Interpreting the Role of Sample Difficulty in RLVR for LLMs
- Reinforcement Learning for Reasoning in Large Language Models with One Training Example
- Sharpening Tax in Post-Training
- STaR: Bootstrapping Reasoning With Reasoning
- Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?
- Planted in Pretraining, Swayed by Finetuning: A Case Study on the Origins of Cognitive Biases in LLMs