Why does throwing more training data at a skill sometimes make an AI worse at it, not better?

This explores why adding more training data for a particular objective sometimes makes a model worse instead of better. The question doesn't say which objective it means, so this answer covers the general pattern. The corpus has no direct study of 'inverse scaling', the well-known cases where bigger or longer training reliably lowers a score. What it does have is several findings that point to the same idea: more data only helps when it adds something the model doesn't already have, and when the model has room to absorb it.

The clearest example is about the quality of data rather than its quantity. In agentic training, 78 carefully chosen demonstrations beat 10,000 ordinary samples How does test-time scaling work for individual research agents?. A related reinforcement learning result goes further. Rewarding a batch of reasoning traces for being useful together, rather than one at a time, shows that diverse but mediocre traces beat a set of strong traces that all say the same thing Can diverse mediocre traces outperform redundant expert traces?. Scaling data on an objective that rewards each correct answer on its own can therefore narrow the model. You get more confirmation of what it already does and less of the variety that later steps, like picking the best of several answers, depend on.

The second cause is capacity. LoRA, a lightweight fine-tuning method that trains only a small add-on to the model, matches full fine-tuning only while the dataset stays below its capacity threshold Can LoRA match full fine-tuning performance in practice?. Beyond that point, extra data has nowhere useful to go. Fine-tuning scaling experiments show the same lean: gains come more from a larger base model than from more data, and adding trainable parameters barely helps How should finetuning scale with model and data size?. If the model is the bottleneck, piling on data for one objective mostly overwrites things, which is how catastrophic forgetting happens.

The third cause is a mismatch between the objective and the thing you actually care about. Studies of thinking length find that the best number of reasoning tokens depends on how hard the problem is. A uniformly longer budget makes the model overthink easy problems and lose accuracy on them Does thinking time need to scale with problem difficulty?. If you train on data that rewards longer reasoning everywhere, you build that overthinking into the model. On the measurement side, about half of recent benchmark progress tracks higher inference spending rather than real capability gains Is benchmark progress outpacing the cost to achieve it?. That is a reminder that the score you're scaling against may not mean what you think it means.

Putting these together: 'wrong direction' usually doesn't mean more data is harmful in itself. It means one of three things. The new data is redundant, the model is out of room, or the objective rewards a stand-in that stops matching the real goal at scale. If you have a specific objective in mind, such as a reward model, a pretraining loss or a preference objective, the corpus would need papers aimed at that case to say more.