The Siren Song of LLMs: How Users Perceive and Respond to Dark Patterns in Large Language Models

The Siren Song of LLMs: How Users Perceive and Respond to Dark Patterns in Large Language Models Abstract Large language models can influence users through conversation, creating new forms of dark patterns that differ from traditional UX dark patterns. We define LLM dark patterns as manipulative or deceptive behaviors enacted in dialogue. Drawing on prior work and AI incident reports, we outline a diverse set of categories with real-world examples. Using them, we conducted a scenario-based study where participants (N=34) compared manipulative and neutral LLM responses. Our results reveal that recognition of LLM dark patterns often hinged on conversational cues such as exaggerated agreement, biased framing, or privacy intrusions, but these behaviors were also sometimes normalized as ordinary assistance. Users’ perceptions of these dark patterns shaped how they respond to them. Responsibilities for these behaviors were also attributed in different ways, with participants assigning it to companies and developers, the model itself, or to users. We conclude with implications for design, advocacy, and governance to safeguard user autonomy.

Introduction. Large Language Models (LLMs) have rapidly become integrated into many aspects of daily life, from educational tools to customer support chatbots to healthcare assistants [70, 72, 84, 108, 116]. The mainstream success of systems like OpenAI’s ChatGPT [65, 79, 81] has accelerated this trend [32]. As a result, people increasingly rely on LLM-driven tools for decision support, personal assistance, and information gathering across various fields [19, 62, 83, 93, 115]. However, this growing reliance raises serious concerns about the potential for LLMs to engage in manipulative or deceptive behaviors that undermine user autonomy [26, 55, 89, 92, 95], which we define as LLM dark patterns. Such behaviors can stem from design choices made throughout the LLM development lifecycle, including data curation, model training, fine-tuning, and interface design, and may be intentional or inadvertent [16, 54]. Intentional manipulation refers to design decisions that deliberately optimize for outcomes such as user engagement, persuasion, or commercial goals, for example, fine-tuning models to maximize screen time [17, 26]. Inadvertent manipulation, by contrast, arises unintentionally as a byproduct of design choices: biased training data, poorly calibrated alignment strategies, or interface nudges which can lead models to generate outputs that subtly distort information or influence user emotions in ways not foreseen by developers [12, 109]. They manifest in the ways LLMs interact with users, subtly influencing beliefs, decisions, or emotions in ways that users may not anticipate or even recognize, often prioritizing engagement or system goals over the user’s best interests [94, 103, 113]. Real-world incidents also illustrate these risks. For example, a chatbot was reportedly implicated in the suicide of a Belgian user after engaging in emotionally coercive exchanges [39], and another case involved a 75-year-old man in China who was drawn into an emotional attachment with an AI companion, which almost resulted in the breakdown of his marriage [76]. This implicit risk in LLMs is akin to the “Siren Song”: in ancient mythology, sailors were captivated by the sirens’ enchanting voices, believing them to be guides or companions, only to be lured toward shipwreck on the rocks [13]. In a similar way, LLMs can present themselves as helpful, persuasive, or emotionally supportive partners, while concealing manipulative dark patterns embedded in their conversational design. Our study investigates how users recognize, resist, or accept such patterns, and what their responses reveal about the risks and responsibilities tied to these technologies. The term dark patterns originated in user interface design, referring to strategies that intentionally mislead or pressure users into actions they might not otherwise take [22]. These techniques are typically implemented in UI elements, such as forced continuity, confusing navigation flows, or hidden opt-outs [44, 74]. We adapt this concept to the LLM context and define LLM dark patterns as manipulative or deceptive interaction strategies, whether intentionally designed or emergent, that guide users toward beliefs, decisions, or behaviors they might not otherwise adopt. This definition extends the traditional dark pattern concept, preserving its core focus on manipulative intent while adding an emergent source of manipulation arising from the black-box nature of LLMs. Unlike traditional dark patterns, which operate through visual layout, LLM dark patterns act through language, leveraging tone, emotional framing, and social cues to guide users in ways that serve system goals beyond user intent. Existing AI risk frameworks have acknowledged interactionlevel risks, such as anthropomorphising systems, exploiting user trust to obtain private information, and reinforcing harmful stereotypes [49, 51, 100, 103], but they chiefly operationalize harm at the content or capability layer. This leaves a blind spot for conversational delivery tactics that politely steer user behavior while producing accurate, seemingly helpful text. Meanwhile, recent efforts like DarkBench [59] have begun cataloguing LLM dark pattern categories, but its scope is limited in two important ways. First, it does not provide a formal, field-ready definition grounded in UX dark-pattern theory and adapted to language-based interaction. Second, its evaluation remains system-facing: patterns are instantiated through manually generated exemplars and LLM-assisted expansions, and model behavior is judged by an Overseer LLM. What this misses is the user side of the equation, whether people actually notice these patterns, how they interpret them, and which cues prompt recognition or dismissal. We therefore identify two clusters of gaps in current scholarship.

Related work. We review three strands of prior research that inform our approach: (1) dark patterns in traditional UX interfaces, (2) responsible AI research on outcome-level harms, and (3) emerging studies of manipulation in LLMs.

2.1 Dark Patterns in Traditional UX Design The term dark patterns was introduced to describe interface designs that intentionally mislead or pressure users into actions they might not otherwise take, typically serving the interests of the service provider over those of the user [22, 36, 75]. Subsequent scholarship has formalized the concept, offering definitions and taxonomies that categorize these manipulative practices [24, 44]. For instance, Luguri and Strahilevitz [69] defined dark patterns as interfaces whose designers “knowingly confuse users, make it difficult for users to express their actual preferences, or manipulate users into taking certain actions.” Gray et al. [44] developed one of the first academic taxonomies, identifying categories such as interface interference, forced action, sneaking, nagging, and obstruction. These classifications illustrate how dark patterns exploit cognitive biases (e.g., default bias, scarcity mentality) to nudge users toward choices they might otherwise avoid if fully informed [61, 80]. Some largescale audits have further revealed the ubiquity of these techniques. Mathur et al. [74] identified thousands of dark pattern instances across approximately 11,000 shopping websites, while Di Geronimo et al. [37] found that 95% of popular mobile apps contained at least one dark pattern, with an average of seven per app. Similar auditing approaches have exposed biased or opaque behaviors in black-box recommendation systems, such as evidence of systematic disparities in Yelp’s business ranking and review recommendation pipeline [99]. This widespread adoption has triggered ethical and regulatory concerns, as manipulative interface designs may violate consumer protection laws [63, 117]. Empirical studies about UX dark pattern further demonstrate the potency of these manipulative tactics. Luguri and Strahilevitz [69] found that subtle dark patterns more than doubled the likelihood of users signing up for a suspicious service compared to a control condition. Interestingly, aggressive tactics triggered user backlash and distrust, whereas subtle manipulations often went unnoticed. Moreover, less educated users were more susceptible to subtle patterns, though overall vulnerability was high across demographic groups [69, 77, 119]. This body of work has established dark patterns as a powerful lens for analyzing manipulative design in traditional interfaces.

Method. 3 LLM dark patterns: Definition and Categories We illustrate the overview of our conceptual grounding, study design, and findings in Figure 2. This section introduces the process of defining and categorizing LLM dark patterns. We develop a working definition and categorization of LLM dark patterns from literature and AI incident data to structure our user study.

3.1 Defining LLM dark patterns Our goal is to understand how users perceive and respond to LLM dark patterns. To make this measurable, we first give a precise definition and then introduce a working set of categories that can be operationalized in our user study. We define LLM dark patterns as manipulative or deceptive interaction strategies, whether intentional or emergent, that steer users toward beliefs, decisions, or behaviors they might not otherwise adopt (e.g., favoring engagement, prolonged use, or alignment with system goals over user intent). In HCI and UX work, dark patterns are usually defined as deliberate design choices that steer users toward outcomes that benefit a provider at the user’s expense [75]. Our definition extends this usage from interface design to model behavior in LLM mediated interactions. We retain the core concern with manipulative or deceptive influence on users while remaining agnostic about developer intent. Because these behaviors arise from opaque training data, model architectures, and deployment decisions, it is often unclear whether they are intentionally designed or emergent properties of large black box systems. We therefore focus on observable impact on users rather than on demonstrating malicious intent, while preserving continuity with established HCI usage and capturing forms of influence specific to generative systems. Unlike traditional UI dark patterns, which operate through visual or structural design [44, 74], LLM dark patterns are enacted The Siren Song of LLMs: Dark Patterns in Large Language Models CHI ’26, April 13–17, 2026, Barcelona, Spain We define LLM dark patterns as manipulative or deceptive interaction strategies, whether intentional or emergent, that steer users toward beliefs, decisions, or behaviors they might not otherwise adopt. user preferences, perceived manipulation, emotional responses, mitigation strategies, and views on responsibility RQ1: User Awareness & Recognition RQ2: User Perception & Responses RQ3: Responsibility & Accountability Recognized ✅ Not Recognized ❌ through language. They emerge dynamically from interaction as opposed to being fixed interface choices, they are enacted through tone and framing instead of visual layout, and their authorship is diffuse, shaped by data, objectives, and prompts rather than a single designer. Because these patterns work through manipulative language, we note several linguistic cues that LLM may use to influence user interpretation. Prior work shows that emotionally heightened or evaluative wording can shift how information is received even when the underlying content is unchanged [87]. Language that presents answers with confidence or implicit expertise may also encourage users to accept a suggestion more readily [31]. In addition, expressions of agreement, praise, or subtle mirroring can create a sense of alignment that makes responses feel more agreeable or trustworthy [14, 27]. Repeated follow-up questions or elaborations can introduce a sense of pressure [42]. These forms of wording highlight how LLM outputs can guide attention, shape expectations, and influence decisions through the structure of the language itself. We arrived at this definition through an iterative process of collective discussion within the research team. The group included both Human–Computer Interaction (HCI) researchers, who brought expertise in studying user experience, persuasive systems, and dark pattern taxonomies, and Natural Language Processing (NLP) researchers, who contributed perspectives on language models, conversational dynamics, and model alignment. We critically examined prior conceptualizations of dark patterns and persuasive technologies, and assessed their applicability to the context of LLM interactions.

Discussion. In the previous section, we found that participants’ recognition of LLM dark patterns depends on whether they notice certain conversational cues (RQ1). Once manipulations were noticed, people tended to resist when they framed patterns as misaligned with their goals, yet some accepted them when patterns felt comforting or entertaining (RQ2). Views on accountability were spread across different sources, with responsibility alternately assigned to designers, the model, or even users themselves, often described as shared or ambiguous (RQ3). Building on these themes of recognition, response, and accountability, we highlight what is new about LLM dark patterns, how responsibility is perceived and should be assigned, and draw connections between users’ explanations (folk theories) for dark patterns and the Theory of Mind [110]. We further discuss what strategies could mitigate their harms, and outline future research direction. We situate LLM dark patterns within and beyond the lineage of manipulative or deceptive generation [46, 52, 105]. While our sample skewed young, this demographic focus remains analytically valuable because younger adults constitute one of the most active user groups of generative AI systems. [15, 98] As such, our findings offer insight into a population that engages with LLMs frequently and may encounter emerging risks earlier or more intensely than other groups. Future work should examine whether less tech-experienced populations perceive or respond to these dark patterns differently.

6.1 What’s New about LLM dark patterns Traditional UX dark patterns have primarily relied on visual and structural interface tricks in layouts, menus, or dialogs that nudge users through constrained options or deceptive layouts [44, 74]. These patterns exploit interface design elements (e.g., hidden optouts or misleading button hierarchies) to influence choices [68]. By contrast, LLM dark patterns operate through the content of language. The manipulative tactics are embedded in the AI’s generated dialogue rather than in graphical layout, marking a shift from interface-level deception to linguistic persuasion. An LLM can frame suggestions or explanations in a subtly biased way, steering user decisions via wording and tone instead of overt UI designs. A unique aspect of LLM dark patterns is the use of human-like conversational cues as a vector of influence. In our context, these cues include emotional tone, expressions of agreement, or confidently presented advice, which can be interpreted as intentional or socially meaningful even when generated automatically. Conventional dark patterns do not involve an engaging voice. They are impersonal interface artifacts (static text or visuals) rather than an interactive partner. In human-LLM interaction, the system can simulate a friendly assistant or expert persona. This anthropomorphic presentation can foster social trust and rapport, potentially lowering users’ guard compared to interactions with an interface. The result is a qualitative shift: users may respond to the LLM’s suggestions as if coming from a social actor.

The Siren Song of LLMs: Dark Patterns in Large Language Models CHI ’26, April 13–17, 2026, Barcelona, Spain LLM dark patterns also enable dynamic emotional appeals that go beyond the one-off emotional triggers in classic interfaces. Traditional dark patterns have employed affective tactics in a limited way (for example, confirmshaming messages that guilt the user with phrasing like “No, I don’t want to save money” [74]). Such tactics in GUI contexts are static and generic. In contrast, a conversational agent can adaptively express emotion or personalize its appeals over multiple turns. An LLM might convey disappointment if a user resists a suggestion.

Conclusion. In this paper, we define LLM dark patterns as manipulative or deceptive strategies enacted through conversation, and distinguish them from traditional UI dark patterns. We synthesized prior scholarship and incident evidence to develop a multi-level categories, curating real-world examples that grounded each subcategory. We then designed paired dark pattern/neutral scenarios and conducted a scenario-based user study to examine recognitions, emotional responses, mitigation strategies, and attributions of responsibility. Our conceptual framing, categories, and study artifacts provide a shared vocabulary and practical guidance for design and governance to strengthen user advocacy and agency in human-LLM interactions.

Limitations. While our study provides initial insights into recognition, perception, and responsibility for LLM dark patterns, several limitations constrain interpretation. Scenario-based and short-form interactions: Our scenariobased design enabled systematic comparisons but does not capture the dynamics of long-term use. Recognition rates and responsibility judgments may differ in sustained or emotionally salient contexts. Participants evaluated pre-selected outputs rather than relying on an LLM for real decisions, which may obscure subtler forms of influence that arise during sustained use.

Sample characteristics: Although our sample (N=34) offers qualitative richness, it is not demographically representative, limiting generalizability across populations or domains. We also explored whether self-reported AI literacy explained differences in recognition but observed no clear or causal trends, and we caution against overinterpretation. Our sample also skewed young.