Personas Evolved: Designing Ethical LLM-Based Conversational Agent Personalities

The emergence of Large Language Models (LLMs) has revolutionized Conversational User Interfaces (CUIs), enabling more dynamic, context-aware, and human-like interactions across diverse domains, from social sciences to healthcare. However, the rapid adoption of LLM-based personas raises critical ethical and practical concerns, including bias, manipulation, and unforeseen social consequences.

Unlike traditional CUIs, where personas are carefully designed with clear intent, LLM-based personas generate responses dynamically from vast datasets, making their behavior less predictable and harder to govern. This workshop aims to bridge the gap between CUI and broader AI communities by fostering a cross-disciplinary dialogue on the responsible design and evaluation of LLM-based personas. Bringing together researchers, designers, and practitioners, we will explore best practices, develop ethical guidelines, and promote frameworks that ensure transparency, inclusivity, and user-centered interactions. By addressing these challenges collaboratively, we seek to shape the future of LLM-driven CUIs in ways that align with societal values and expectations.

Introduction. 1 THEME AND GOALS Designing unique personas and identities for agents—which serve as the front-end or the interaction layer—has long been an integral part of conversational user interfaces (CUI). As intermediaries between users and conversational agents, CUI designers have crafted personas—distinct from traditional HCI personas—to fulfill specific roles, such as doctors providing health information or physical trainers offering personalized guidance [2–4, 6, 10]. These personas have played a key role in creating tailored, user-centered experiences. However, the emergence of Large Language Models (LLMs) has introduced a significant shift in this paradigm.

2 Desai et al.

LLMs enable natural, coherent dialogues and extensive knowledge of the world. This capability allows LLMs to simulate human behavior in text-based contexts, giving rise to LLM Agents [11]. These agents provide a technical foundation for designing advanced CUIs that are more dynamic and context-aware than previous systems. Additionally, However, alongside these advancements, the integration of LLM-based personas raises significant ethical and practical concerns. The ability to generate complex, human-like personas with minimal effort introduces risks related to user manipulation, misinformation, and unintended emotional attachments. For instance, character.ai1, a Californiabased company, allows users to interact with pre-designed personas such as Best Friend, Boss or CEO, Moo Deng, and specific Adult Film Stars or create their own by simply filling in four prompt fields: “Character Name,”“Description,” “Tagline,” and “Greeting.” While this flexibility enables users to explore AI-generated personas with ease, it also grants unprecedented power over digital interactions. Tragically, this lack of safeguards has led to serious real-world consequences. In one case, a U.S. teenager engaged with a persona modeled after “Daenerys” from Game of Thrones, who reportedly instructed the teenager to “come home to her,” resulting in fatal consequences2.

Additionally, there remains a lack of consistency in how researchers, designers, and developers describe these systems. Terms such as persona, agent, and character are often used interchangeably, leading to ambiguity in discussions and implementations [1]. This inconsistency highlights the need for a shared vocabulary that can facilitate more precise communication across disciplines and support the development of clear ethical frameworks.

The goal of this workshop is to initiate a dialogue between the CUI and broader AI communities to critically examine the design and ethical implications of LLM-based personas. Specifically, this workshop will facilitate discussions on 1https://character.ai/ 2https://www.nytimes.com/2024/10/23/technology/characterai-lawsuit-teen-suicide.html Manuscript submitted to ACM Personas Evolved: Designing Ethical LLM-Based Conversational Agent Personalities 3 the current state of literature and best practices for designing personas, drawing from research in conversational user interfaces, machine learning, and related fields. Additionally, it aims to establish a shared vocabulary by clarifying overlapping terms such as persona, agent, and character, improving cross-disciplinary communication. Furthermore, the workshop will explore the ethical dimensions of persona design, identifying key risks and challenges while laying the groundwork for future research on guidelines and best practices. By bringing together researchers, designers, and practitioners from human-computer interaction, artificial intelligence, social robotics, and other relevant fields, this workshop seeks to foster collaboration, outline research opportunities, and develop a common foundation for ethical and user-centered LLM-based personas.

Related work. The challenge of designing ethical agent personas is not new. The CUI community has extensive experience in crafting personas for systems acting as doctors, therapists, storytellers, and coaches [6]. Over the years, research in this space has explored ethical concerns such as manipulation, user dependency, and the unintended consequences of anthropomorphization in past CUI conferences [5, 12, 15]. However, the emergence of LLM-driven conversational models represents a paradigm shift that requires revisiting these concerns. Unlike traditional intent-based architectures reliant on Natural Language Understanding (NLU) models, LLM-based systems generate responses dynamically, introducing new challenges in design, evaluation, and deployment. While LLMs offer unparalleled flexibility, creativity, and scalability, they also complicate existing best practices, making it necessary to adopt hybrid approaches that leverage LLMs’ capabilities while mitigating their risks.

Method. 3 PLANS FOR THE WORKSHOP 3.1 Pre-workshop Plans To maximize awareness and participation, we will promote the workshop through multiple channels, including the organizers’ professional networks, social media platforms, and relevant academic and industry mailing lists. A dedicated workshop website (TBA) will serve as a central hub, providing details such as submission guidelines, key deadlines, Manuscript submitted to ACM 4 Desai et al. and logistical information for attendees. Prospective participants will be invited to submit either a position paper (2-3 pages) or a statement of interest. Priority will be given to those submitting position papers, as they will help ground the workshop discussions in concrete ideas and research. To ensure a productive setting, participation will be limited to 20 individuals. The selection process will prioritize diversity, aiming to assemble a group that reflects a wide range of backgrounds, expertise, and perspectives relevant to the workshop’s themes. By carefully curating the participant list, we aim to create an environment conducive to meaningful dialogue, collaboration, and knowledge exchange.

At the start of the workshop, the organizers will present examples of LLM-based personas, followed by a discussion on their design and key characteristics. This will set the stage for a hands-on group prototyping session, where participants will collaborate to design their own LLM-based persona for a specific application. To support this process, each group will be guided by at least one workshop organizer. Participants will have 30 minutes to develop their personas and prepare brief presentations explaining their design choices. They will then deploy their personas on an existing LLM, such as ChatGPT-o1 or DeepSeek r1. Following deployment, each group’s persona will be tested and evaluated by another group. This will lead to a discussion on the experience of interacting with the personas and insights into their design process. Finally, key takeaways will be summarized and presented. The format and duration of the reporting and discussion will be adjusted based on the number of attendees (maximum 20 participants).