Do prompting fixes work equally well across all model sizes?

Having found that models are not robust to task-irrelevant persona attributes, the authors "design three alternative prompting methods to guide model behavior more directly than merely including a persona description," then repeat their Expertise Advantage and Robustness experiments with each mitigation strategy applied. Overall, the strategies backfire: "mitigation strategies negatively impact Expertise Advantage and Robustness, as they increase the number of tasks where experts and irrelevant personas reduce performance," a result the paper confirms with a mixed-effects regression reported in its appendix. But "for the largest models (Llama-3.1-70B, Qwen-2.5-72B), the pattern changes: mitigation strategies preserve Expertise Advantage and significantly improve Robustness."

The mitigation strategies are prompting-level interventions layered on top of the persona description, tested across the same nine-model, 27-task setup used for the paper's core desiderata measurements. The split outcome — helping the two largest models while hurting the rest — reads as a capability threshold rather than a general fix: the authors had already established that the robustness failure "is not solved by scaling up" in isolation, so the finding here is that scale combined with explicit mitigation prompting, not scale alone, is what the largest models in the study benefit from.

This is a distinct finding from the headline robustness failure described in the sibling note Do expert personas actually improve LLM task performance?: that note states the problem, this one states which models the paper's own proposed fix can repair and which it cannot. It also complicates the premise in Can layered persona architecture sustain coherent character behavior? that richer persona architecture can stabilize persona behavior: here, a more elaborate prompting intervention made robustness worse for most models in the study, and only helped the two largest.

The paper tests mitigation at the largest scale only on two models, Llama-3.1-70B and Qwen-2.5-72B, so "largest, most capable" is defined relative to this nine-model, three-family set rather than against proprietary or larger models not tested here; the excerpt does not establish where between these sizes the threshold between mitigation helping and mitigation backfiring actually sits. The implication the authors draw is narrow: robustness fixes for smaller or mid-sized open-weight models need an approach other than the mitigation strategies tested, since scaling those same strategies down does not transfer the benefit.