When Persona Attributes Improve Population Alignment in Large Language Models

Large Language Models (LLMs) are increasingly used to predict the responses of human participants in survey panels. Towards that goal, persona prompting has recently emerged as a technique to inform and align large pretrained language models. Persona prompting refers to the practice of using short textual descriptions of ’personas’ in prompts to steer the LLM’s generations. Personas describe individuals through different attributes such as their socio-demographics, attitudes, or behaviors, with the aim of aligning LLMs to produce responses that correlate with the corresponding human responses. Yet, recent work has produced mixed and partly conflicting results of persona prompting without clear patterns of success and failure. Among the few consistent findings is that the selection of persona attributes matters, and that using more attributes does not necessarily lead to better performance. It remains unclear how different attribute selection methods perform and how to choose among them. In this paper, we propose that observed human response variation of a survey question is a potential explanation for the mixed performance observed so far. In addition, we compare the performance of persona prompting associated with different methods for selecting persona attributes.

Introduction. The use of personas to introduce diversity and simulate different human perspectives is rapidly gaining traction in Natural Language Processing (NLP) and Computational Social Science (CSS). By conditioning large language models (LLMs) on demographic, attitudinal, or behavioral characteristics, persona prompting promises a scalable approach to modeling heterogeneous viewpoints and social behavior, often combining empirical observations with synthetic data generation. Despite this promise, empirical findings remain mixed. While some studies report substantial improvements in the representation of diverse perspectives, others find limited or inconsistent gains, raising fundamental questions about when and why persona prompting succeeds. In particular, the conditions under which personas enable LLMs to faithfully reproduce human judgments and behaviors remain poorly understood. The limited empirical evidence available on the sources of these mixed results comes primarily from subjective annotation tasks. In this context, Brown et al. (2025) and Sarumi et al.

Discussion / Conclusion. and Implications For practitioners interested in simulating survey responses via persona prompting, our results suggest a two-step framework for practical use. In a first step, our measures of human response variation help identify types of survey questions for which the simulation can be expected to work best. As we have shown above, this will generally be the case for questions with high high human response variation, i.e., questions (and potentially tasks and topics) that are highly contested. In a second step, our results suggest to use approaches informed by existing survey data for the same or a similar question to identify the attributes that are most important for persona creation and lead to the best alignment between predicted and actual responses.