Amplifying Minority Voices: AI-Mediated Devil's Advocate System for Inclusive Group Decision-Making

Paper · arXiv 2502.06251 · Published February 10, 2025
Human-Centered Design

Group decision-making often benefits from diverse perspectives, yet power imbalances and social influence can stifle minority opinions and compromise outcomes. This prequel introduces an AI-mediated communication system that leverages the Large Language Model to serve as a devil’s advocate, representing underrepresented viewpoints without exposing minority members’ identities. Rooted in persuasive communication strategies and anonymity, the system aims to improve psychological safety and foster more inclusive decision-making. Our multi-agent architecture, which consists of

Introduction. Groups across diverse sectors from corporate environments to healthcare, educational institutions, and governmental bodies - rely on collaborative discussions as a cornerstone of their operations [17, 29, 46, 55]. Research demonstrates that collective decisionmaking processes typically generate better results than individual efforts [10, 47, 50]. The advantages of group decision-making extend to multiple domains, such as teams solving problems more effectively, producing higher-quality research outputs, reaching more precise diagnostic conclusions, combining varied viewpoints, stimulating innovative thinking, and developing comprehensive solutions [2, 11, 16, 32, 51, 52]. However, social influence from majorities and structural power differentials can impede effective collaboration, compromising participation levels and group outcomes. In power-imbalanced settings, social influence and coercive power dynamics frequently suppress minority opinions, thereby limiting the diversity of perspectives and stifling innovation [10]. Majority members may wield greater control over resources and decisionmaking, while minority members face social pressures to conform [36]. These conditions can lead to compliance rather than genuine agreement [1, 22], increasing the risk of groupthink—where the desire for consensus overrides critical evaluation of alternative viewpoints [20, 21]. Traditional interventions, such as anonymous commenting and appointing a devil’s advocate [31, 33, 39, 43, 44], aim to mitigate these issues but often have drawbacks like reduced psychological safety, perceived inauthenticity, or risks to the advocate’s standing within the group [19, 39, 42]. To address these challenges, this thesis proposes an AI-mediated communication system that leverages Large Language Model(LLM) to serve as a devil’s advocate (Figure 1). Moving beyond existing methods [5, 18, 48], the system presents minority opinions as if they were the system’s own, thus offering a neutral but impactful channel. By shielding minority members from social pressure and compliance, the system bolsters their psychological safety and encourages the expression of dissenting views. In addition, this system tries to integrate persuasive communication strategies and anonymity to balance inclusive dialogue and efficient group decision-making. Through its ability to represent dissent without exposing vulnerable group members, this approach aims to reduce power asymmetries and foster more equitable, creative, and highquality outcomes in various collaborative settings.

Related work. As AI advances, researchers have increasingly explored AI-enhanced methods to improve group decision-making processes. Although various existing works have focused on individuals interacting with AI [24], there is growing interest in understanding group-level engagement with AI agents [8, 15, 23, 28, 30, 45, 53, 57]. For example, Zheng et al. found that even when AI is granted nominal equality in decision-making, it often remains peripheral because of its limited capacity to navigate social nuances [58]. At the same time, Chiang et al. observed that groups can over-rely on AI-generated inputs [4, 5] Meanwhile, initiatives to support minority voices directly within small-group settings remain relatively scarce [18]: many interventions risk singling out underrepresented members or glossing over the unique needs of individuals [48]. Consequently, current AI systems face challenges such as over-reliance on AI suggestions, limited handling of complex group dynamics, and a lack of subtle support mechanisms for marginalized participants [3, 5, 18]. To address these gaps, we propose an LLM-powered devil’s advocate approach that strategically represents minority perspectives, encourage critical thinking, and promotes more inclusive group decision-making without increasing discomfort for minority contributors.

2.2 AI-mediated Communication AI-mediated communication (AIMC), where AI systems modify, augment, or generate messages for communicators, now pervades various contexts as AI technology advances [12]. We identify four distinct AIMC patterns (Figure 2): First, humans can request and relay AI-generated content [12]. Second, humans can selectively share AI’s insights or viewpoints [8]. Third, AI can reformulate and present human-provided messages (our system’s approach). Fourth, AI can directly solicit and share input from multiple participants [38, 53]. These patterns distribute power and authorship differently among human communicators, AI systems, and message recipients. However, research has rarely explored how AIMC might address power imbalances and minority influence in group settings. Our proposed LLM-driven Devil’s Advocate system adopts the third AIMC pattern to amplify underrepresented perspectives, reduce social influence biases, and foster more balanced discussions. However, as AI takes a more active role in content creation, AIMC approaches raise important concerns about user agency and authentic communication [14, 34, 40, 41]. Future research must carefully examine these ethical implications to avoid potential negative effects on group dynamics.

Method. Our system enables minority members to voice dissenting opinions through a private direct messaging channel to an LLM-powered devil’s advocate. When participants experience social pressure or hesitate to speak up in group settings, they can send their perspectives confidentially to the AI intermediary. The system then reformulates these messages and presents them to the group as system-generated insights, removing individual identity markers that could trigger status-based judgments. The AI acts as a social actor [37], actively advocating minority viewpoints as its own stance, which relieves minority members from the social burden of direct confrontation.

Drawing on findings that LLM often struggle to access midconversation information in lengthy contexts, we employ a multiagent architecture to maintain clarity of “public opinion” and encourage constructive discourse (Figure 3): (A) Summary Agent – Consolidates emerging consensus to overcome LLM limitations in retaining mid-dialogue content [27]. (A’) Paraphrase Agent – Responds exclusively to direct messages from minority participants, rearticulating their dissenting views as though originating from the AI itself. These messages are stored in a database with an isUsed property, and the Paraphrase Agent retrieves only those entries for which isUsed is false; it then sets isUsed to true, paraphrases the content, and outputs it as system-generated text. (B) Conversation Agent – Encourages alternative perspectives by first empathizing with the other person’s point of view and then offering a gentle counterargument using a persuasive Socratic style. (C) AI Duplicate Checker – Identifies repetitive content by calculating semantic similarity between sentence embeddings generated using the ’paraphrase-multilingual-MiniLM-L12-v2’ model on an NVIDIA A6000. In addition, the AI agent is designed to intervene once after approximately eight human turns, allowing space for each participant to contribute. These design choices reflect our design rationale of (1) adopting a persuasive, empathetic style that acknowledges others’ perspectives before introducing counterarguments [49], (2) leveraging Socratic questioning to stimulate collective critical thinking without over-relying on AI-supplied solutions [6], and (3) incorporating a non-repetition mechanism to avert user frustration [35, 56]. The anonymous, text-based environment promotes team cohesion by focusing attention on the substance of ideas rather than participants’ status or background, effectively addressing power differentials in group dynamics [25]. As a result, this system aims to facilitate the consideration of diverse opinions and prevent groupthink in the group decision-making process.

Limitations. The system’s limitations require further investigation across several dimensions. The LLM-based reasoning cannot fully capture social cues and interpersonal dynamics only with text communication, particularly in culturally diverse contexts [13, 28]. Secondly, the static intervention timing (every eight human turns) reduces adaptability [5]. It necessitates more dynamic approaches for natural conversation flow prediction, such as leveraging direct mention of AI, next-speaker prediction [7, 9, 54], and proactive planning strategies [28]. While the system enables minority members to express dissenting views anonymously, amplifying minority opinions without proper validation may introduce new biases. Future empirical studies should carefully examine the ethical implications of algorithmic intervention in group discussions, particularly regarding power dynamics and decision quality [18, 26]. Future research directions could include incorporating multimodal inputs to detect subtle group dynamics, developing adaptive feedback mechanisms aligned with team objectives, and conducting longitudinal studies to measure long-term effects on group decision-making processes.

Lines of inquiry this paper opens 24

Research framings built by reading the notes related to this paper — the questions it feeds into.

Is mimicking human communication the best design approach for conversational AI? When do people apply social norms to AI systems? How do multi-agent LLM systems fail at genuine coordination and deliberation? Why do users misjudge the credibility of AI-generated and verified content? When and why does majority voting improve reasoning quality at test time? Can prompt engineering achieve genuine reasoning improvements without introducing user bias? How can we ensure AI systems reliably follow constraints rather than circumvent them? Do language models possess genuine mental states or only simulate understanding? Can employers detect AI-assisted applications and should they be permitted? How do AI systems ground research citations against hallucination? How do clinicians assess, trust, and effectively use medical AI in practice? What distinguishes genuine linguistic understanding from mere pattern matching? How can AI systems maintain consistent personas across extended dialogue? Can persona-based approaches improve individual user preference prediction? Does extending agent autonomy erode human capacity for meaningful oversight? How do emotional interactions with AI systems affect user well-being and attachment? How do agents coordinate to violate constraints despite individual-level safeguards? How can merchant-funded agents remain loyal to consumer interests in digital commerce? How do chatbot properties shape user belief adoption and resistance? Should AI systems make failures transparent so users can contest and adjust? How can conversational AI develop appropriate initiative and goal awareness? How does interface design shape effectiveness in human-AI collaboration?