ProSA: Assessing and Understanding the Prompt Sensitivity of LLMs

Large language models (LLMs) have demonstrated impressive capabilities across various tasks, but their performance is highly sensitive to the prompts utilized. This variability poses challenges for accurate assessment and user satisfaction. Current research frequently overlooks instance-level prompt variations and their implications on subjective evaluations. To address these shortcomings, we introduce ProSA, a framework designed to evaluate and comprehend prompt sensitivity in LLMs. ProSA incorporates a novel sensitivity metric, PromptSensiScore, and leverages decoding confidence to elucidate underlying mechanisms. Our extensive study, spanning multiple tasks, uncovers that prompt sensitivity fluctuates across datasets and models, with larger models exhibiting enhanced robustness. We observe that few-shot examples can alleviate this sensitivity issue, and subjective evaluations are also susceptible to prompt sensitivities, particularly in complex, reasoningoriented tasks. Furthermore, our findings indicate that higher model confidence correlates with increased prompt robustness. We believe this work will serve as a helpful tool in studying prompt sensitivity of LLMs. The project is released at: https://github.com/ open-compass/ProSA.

Introduction. In recent years, large language models (LLMs) have rapidly become the focus of the artificial intelligence field. By training on large-scale corpora, LLMs have shown impressive capabilities in multiple tasks (Zhao et al., 2023; Min et al., 2023). The input for LLMs, known as prompts, is crucial for their ability to complete a wide variety of tasks. The prompts for LLMs come in various forms. Even for the same requirement, different individuals’ varying expression habits can result in prompts of different styles. Figure 1 illustrates four The diversity of prompts elicits various responses from LLMs. Recent studies (Zhu et al., 2023; Pezeshkpour and Hruschka, 2024; Sclar et al., 2024; Zhou et al., 2023) investigate model performance across different prompt templates and demonstrate that LLMs are highly sensitive to the nuances of prompts. Even minor alterations to prompts can lead to substantial declines in model performance. This sensitivity to prompts poses a challenge for researchers aiming to precisely evaluate the models’ capabilities.

Discussion / Conclusion. In summary, we propose an instance-level prompt sensitivity metric, PSS, and conduct a comprehensive analysis on both objective and subjective evaluation. Additionally, we explore the relationship between prompt sensitivity and model confidence. We believe our work can provide guidance for further sensitivity analysis and building robust LLMs.