Prompt-based fixes for AI misbehavior work great on huge models but backfire on smaller ones — so where's the cutoff?

This explores whether there's a specific model size where fixes like mitigation prompts stop working, and what actually sets that line. The direct answer is that the corpus can't give you an exact number. The clearest evidence comes from a study of persona robustness. It tested three prompting-based fixes, and they made only the two largest models more robust (Llama-3.1-70B and Qwen-2.5-72B). Every smaller model got worse Do prompting fixes work equally well across all model sizes?. So the threshold sits somewhere between the mid-sized models tested and roughly 70B parameters. The study didn't test enough sizes in between to narrow it down. Its main point isn't the number. Scale alone didn't fix the problem, and neither did the intervention alone. Only the two together worked.

The more useful finding is that the threshold seems to depend on the kind of fix, not just the size of the model. A prompting fix asks the model to read an instruction about its own behavior, understand it and apply it. That is a demanding skill, and smaller models seem to get confused by it rather than helped. When the fix is built into training instead, the size line drops sharply. A 3B model reached frontier-level math and coding scores through careful post-training design Can small models match frontier reasoning without massive scale?. A 9B model trained on the real results of its edits beat much larger prompted models at patching agent harnesses Does training editors on real outcomes beat prompting larger models?. In that comparison, the prompted models were optimizing for edits that looked plausible, while the trained model checked whether its edits actually worked.

Another lever is to change what surrounds the model instead of the model itself. Giving smaller models more compute at answer time can match larger models on hard prompts Can inference compute replace scaling up model size?. Better execution harnesses can lift frozen models without retraining Can execution harnesses lift model performance without retuning weights?. A committee of weak models can match a strong one, but only with an external checker such as tests, proofs or type checks to pick the right answer When can weak models match strong model performance?. In each case, the smaller model gets help from something outside itself, so it doesn't need to understand its way to the fix.

Two warnings complicate the simple idea that bigger is better. First, some fixes fail for reasons that have nothing to do with size. The same security instructions cut risky exposure by 40–75% for single agents but did nothing in multi-agent systems, because the agent taking the risky action didn't hold the warning Why does prompt hardening work for single agents but not multi-agent systems?. Second, scale can work against you. For generating varied synthetic data, models around 500M parameters produce more distinct outputs per sample than larger ones, which tend to repeat their favorite answers Why aren't bigger models better for generating diverse outputs?.

The takeaway: asking where the size threshold is turns out to be the wrong question. It is better to ask whether a fix depends on the model understanding an instruction. If it does, expect to need something like 70B-class models. If the fix is trained in, checked by an external signal or built into the surrounding system, models as small as 3B–9B can benefit.