If you clean up the meaning of your AI's training instructions, do small and large models benefit equally?

This reads the question as: if you audit instruction data for whether it means what it should, do small and large models gain equally? The corpus has no note that runs that experiment across model sizes, so there is no direct answer. The adjacent evidence points away from "uniform". It also suggests that checking meaning may not be the main lever.

Semantics may matter less than you'd expect. One study trained models on semantically empty or deliberately incorrect instructions. They performed comparably to models trained on the correct ones (about 43% against a 42.6% random baseline). The authors conclude that what transfers is knowledge of the output space, not understanding of the task (Does instruction tuning teach task understanding or output format?). If that holds broadly, an audit that fixes the meaning of instructions has little to work with. That is one setup, though, and it says nothing about whether bigger models behave differently.

Where data quality clearly matters is in which examples you train on. LESS uses gradient features to pick the 5% of instruction data most similar to a target capability, and that 5% consistently beats training on everything. The reason is that mixed datasets contain examples that actively hinder a skill by pulling reasoning strategy away from the task (Can we train better models on less data?). That is influence-based selection, not a read-through for correct meaning. The cheap supply of instruction data also changes what an audit is auditing. Aligned models can generate their own instructions from formatting tokens alone, and 4M such pairs matched human-curated data in quality (Can aligned LLMs generate their own training data?). The summaries provided don't break any of these results out by model size.

There are reasons to expect the effect to vary with size, though this is inference, not a measured result. Models respond to instructions differently at different scales. Small models lose instruction-following linearly as instruction density rises, mid-range models lose it exponentially, and reasoning models hold up to about 150 instructions before failing steeply (How does instruction density affect model performance?). Larger and instruction-tuned models also lean harder on their own stored knowledge and follow user-stated context less (Do larger models follow stated beliefs less often?). The same cleaned dataset would therefore meet models with different failure curves and different habits of deferring to instructions.

So the best-supported answer is probably not, but nobody in this collection has measured it. The missing test is one audited dataset, one selection method, and several model sizes trained side by side. Until someone runs it, treat any claim of uniform gains from semantic auditing as unsupported.