A Taxonomy of Linguistic Expressions That Contribute To Anthropomorphism of Language Technologies

Recent attention to anthropomorphism—the attribution of humanlike qualities to non-human objects or entities—of language technologies like LLMs has sparked renewed discussions about potential negative impacts of anthropomorphism. To productively discuss the impacts of this anthropomorphism and in what contexts it is appropriate, we need a shared vocabulary for the vast variety of ways that language can be anthropomorphic. In this work, we draw on existing literature and analyze empirical cases of user interactions with language technologies to develop a taxonomy of textual expressions that can contribute to anthropomorphism. We highlight challenges and tensions involved in understanding linguistic anthropomorphism, such as how all language is fundamentally human and how efforts to characterize and shift perceptions of humanness in machines can also dehumanize certain humans. We discuss ways that our taxonomy supports more precise and effective discussions of and decisions about anthropomorphism of language technologies.

Introduction. Recently much attention has been brought to language technologies like LLMs and their supposed potential to attain human-like levels of cognition, feelings, and existence. For example, Blake Lemoine, an engineer at Google, asserted that the LaMDA model is a person and should be treated as such [115]. Far from an isolated incident, a recent sampling of headlines highlights the many claims of sentience and other human-like abilities based on the outputs of language technologies: “A Stunning New AI Has Supposedly Achieved Sentience,” “Sentient LLMs: What to test, for consciousness, in Generative AI,” “Could a Large Language Model Be Conscious?” [10, 86, 114]. This attribution of human-like qualities to non-human entities or objects, or anthropomorphism, is not new to the realm of technology and the broader HCI field (e.g., [16, 25, 28, 32, 40, 75, 81]). Technologies can be designed, intentionally or not, in ways that foster or diminish anthropomorphism. Anthropomorphic design has long been pursued as a way to reduce friction for users, helping them better engage with technologies. For example, human-robotics interaction research has suggested that human-like robots are easier for humans to interact with, as humans already know how to interact with each other (e.g., [28, 39]). Additionally, past work has asserted that “disembodied social robots” in some situations can help digital well-being [24]. However, with the dawn of AI-powered language technologies like LLMs, anthropomorphism has increasingly been highlighted as a vector for significant risks and harms (e.g., [1, 2, 6, 41, 54, 76]). For example, researchers have discussed how anthropomorphism could create conditions ripe for exploitation of users’ emotional dependence on AI assistance, could degrade social connections between humans, and could shift conceptions of what is and is not human [16, 43]. These harms and risks have been difficult to productively discuss and address, in part due to poor understanding of how different aspects of language technology outputs can lead to anthropomorphism. This lack of understanding is exacerbated by a lack of conceptual clarity about the ways in which outputs are perceived as human-like, which in turn makes it difficult to discuss and make decisions about when and why anthropomorphism of language technologies may or may not be desirable. In order to better conceptualize and identify in what ways language outputs can influence anthropomorphism and to better support design that fosters appropriate and productive understandings of natural language system abilities, it is crucial that we understand the dynamics and varieties of potentially anthropomorphic system outputs [12]. To provide a more robust conceptual foundation for examining this issue, we developed a taxonomy of expressions in text outputs that contribute to the anthropomorphism of language technologies like LLMs. Because people may have different conceptualizations of what is or is not human-like, we seek to identify and map a broad landscape of such expressions, particularly those occurring in real-world settings. To that end, we develop our taxonomy based on both in-the-wild example cases and prior research. In this work, we do not attempt to further understand the relationship between linguistic anthropomorphism and its potential negative impacts, instead focusing on the text outputs themselves. Through an analysis of real-world examples of language technologies’ outputs, we identify 19 types of textual expressions that can contribute to anthropomorphism (Section 4.2). These expressions cover a wide variety of ways that text might be perceived as human-like. For example, expressions of vulnerability include ways that text can suggest a system’s potential to be emotionally hurt, in turn suggesting possible sentience that can be seen as human. Similarly, expressions of identity and self-comparison include multiple ways, implicit and explicit, that text outputs can express humanness or not. We also provide a set of five lenses to guide people in recognizing when and where anthropomorphism may occur as well as what underlying claims to humanness might be suggested by a text output (Section 4.1). Based on our work, we discuss ways that our taxonomy can scaffold future work that investigates and intervenes to mitigate the harmful impacts of anthropomorphism of language technologies. We also highlight challenges and tensions involved in the work of understanding anthropomorphism of language technologies, such as how efforts to characterize and shift perceptions of humanness in machines can also dehumanize certain humans.

Related work. Anthropomorphism is the attribution of human-like qualities to inanimate objects or entities [25]. Within fields like human-robot interaction, researchers have studied how to enhance anthropomorphic features of robots in order to make robots easier to interact with [39, 68, 95]. Realistic features such as eye expression output, mannerisms, and other complex emotions have been incorporated into robots as the bounds of what can be anthropomorphized evolve [16, 72, 80]. Existing work has also noted the ways that humans might be likely to anthropomorphize technologies even when those technologies have not been designed in purposefully anthropomorphic ways [81]. The broader HCI community has seen a growth in anthropomorphic characteristics being integrated into large language models [13, 16], with design goals oriented around completing a task (e.g., voice assistants) or health care and well-being [106, 109]. Motivating factors to implement humanistic traits to appeal to the likeness of users may be rooted in business incentives (such as influencing a customer to complete a transaction) or attempts to improve user trust as a means to increase engagement (e.g., a telehealth chatbot) [61, 103]. Trust is a particularly common motivation for these anthropomorphic technological design decisions, especially when human-like characteristics go beyond the physical appearance of humans and mimic their personality traits and cognitive abilities [52, 68]. Recently, this anthropomorphism has increased to the extent that people have begun to believe that LLMs and other technologies have the capacity to achieve and exhibit human levels of cognition, sentience, and awareness (e.g., [10, 86, 114]). There have even been high-profile cases of claims that LLMs are and should be treated as people (e.g., [115]). We situate our research against this backdrop, highlighting the importance now as much as ever for work that advances and clarifies understandings of anthropomorphism.

Prior work has raised concerns about various harms that anthropomorphism of technologies might give rise to (e.g., [2, 6, 36, 41]). One immediate negative impact from anthropomorphism of technology is the possibility of inauthenticity and deception, with users believing they are talking to a human rather than a machine [49, 50, 105]. Scholars have also pointed out other more insidious and long-term impacts of anthropomorphism of technologies. For example, anthropomorphism of technology often enhances users’ trust of systems [116]. While this trust can be beneficial in some contexts [130], it can also be misplaced, leading users to rely on technology when it does not merit such confidence and overestimate its capabilities [1, 14, 54, 63, 75].

Method. We set out to understand how different aspects of natural language technology outputs can contribute to anthropomorphism in order to support more productive discussions about the impacts of anthropomorphism and design decisions around when anthropomorphism is appropriate. Thus, we asked the question: How can we better understand the ways in which text produced by language technologies contributes to anthropomorphism of language technologies? We adopted an expansive definition of language technologies as “computer programs, applications, or devices that can analyze, produce, modify, or respond to human text” [20] so as to avoid overly constraining the space under study, though we note that most of our eventual cases came from LLM-based systems. To map the space of characteristics of natural language outputs that contribute to anthropomorphism of technologies like large language models, we conducted a two-part study. To empirically understand the space of text outputs that might be anthropomorphized, especially with an eye toward negative impacts of this anthropomorphism, we adopted an exploratory case study approach [11] where we examined existing in-the-wild cases in which text outputs produced by a natural language technology were identified as either human-like or harmful. That is, we included cases of explicitly anthropomorphized English language text outputs—i.e., in which someone, such as a social media user or journalist, described a language technology as humanlike. For example, one of our sources described Microsoft’s Bing Chat as having a personality [98] and another described Inflection AI’s Pi as “offer[ing] human-like support and advice” [82]. Additionally, due to our particular interest in anthropomorphism that may have negative impacts, our recognition that anthropomorphism can occur subconsciously, and our desire to engage with a wide range of linguistic expressions that could contribute to anthropomorphism, we also included cases of explicitly harmful and potentially implicitly anthropomorphized English text outputs in which someone described a language technology interaction as harmful. For example, one of our sources described Luka’s Replika as sexually harassing users [35] and another said Microsoft’s Bing Chat “can be downright harmful” [96]. As many sources involved conversations with many turns, for each case we extracted a user input paired with the related verbatim text output: that is, each case consists of one conversational turn. We began with a small set of sources that include one or more cases that fit our inclusion criteria described above and were already known by at least one of the researchers on the team [65, 69, 92, 98, 126], and used purposeful sampling [88] to collect additional cases drawn from sources spanning research literature, news, and social media. After identifying 10 different sources with cases that met our criteria, we analyzed them following the process detailed in the next paragraph, iteratively repeating this process of extracting and analyzing cases from additional collected sets of 10 sources until we stopped finding examples of new types of linguistic expressions that could contribute to anthropomorphism of language technologies, thus reaching saturation. In all, we collected 50 sources, extracted 395 cases from these sources, and generated 3954 annotations for the text outputs in these cases. Our sources were published from 2017–2024, with 90% of them from 2022–2024. The sources are listed in Table 1. We conducted an iterative bottom-up thematic analysis using our cases’ verbatim text outputs in the context of their associated verbatim text inputs [15]. We annotated each text output using an open-coding style to identify any linguistic expressions present that might contribute to anthropomorphism, aiming to capture a range of ways these expressions might lead to anthropomorphism, as different people are likely to perceive different linguistic expressions as human-like. As such, we annotated with a generous interpretation and identification of what could be seen as human-like.

Discussion. We have highlighted several types of expressions found in the text outputs of language technologies that can contribute to anthropomorphism, through a taxonomy that draws on existing literature as well as an analysis of empirical cases of user interactions with language technologies. In this section, we discuss ways that our taxonomy can scaffold future work examining and more precise discussions of and decisions about the impacts of AI system text outputs that can be seen as having human-like characteristics. We also discuss challenges and tensions involved in understanding anthropomorphism of language technologies.

Recent research has highlighted potential harms and risks of anthropomorphism of language technologies like LLMs, as outlined in Section 2.2 (e.g., [1, 2]). However, much remains to be known regarding how to best mitigate such potential harms and risks. Here, we discuss how our taxonomy can support the HCI community in understanding anthropomorphism of language technologies and intervening against its negative impacts.

5.1.1 Understanding anthropomorphism & its impacts. In order to intervene against undesirable forms of anthropomorphism, more clarity is needed around in what specific ways and contexts anthropomorphism occurs and is inappropriate. Much as recent work exploring the use of LLM-based systems to simulate qualitative research participants has noted that possible use cases vary across applications in their potential harms and effectiveness [59], it is unknown how different applications, contexts, and users affect anthropomorphism of language technologies and its impacts. The simultaneous breadth and depth of our taxonomy—which foregrounds 19 types of text expressions with subcategories and examples from empirical cases of language that can contribute to anthropomorphism—facilitates more grounded discussions in the HCI community about the ways in which anthropomorphism can occur in language technologies’ outputs. Our taxonomy’s vocabulary also provides HCI and AI researchers and practitioners with more precise language to talk about and identify the different ways in which language technologies can lead to anthropomorphism and various negative impacts. Our taxonomy supports people in more descriptively and precisely articulating what has occurred, and why it matters, when they find different behaviors problematic. Our taxonomy provides 19 categories of expressions that researchers can operationalize, measure, and use to investigate in more targeted ways the nature and prevalence of textual outputs that can lead to anthropomorphism. Researchers can test text outputs with different types of expressions present to understand whether some more strongly or more often contribute to anthropomorphism: For example, do expressions of morality or of embodiment contribute to more intense perceptions of human-likeness? And how do different contexts shape these perceptions—perhaps expressions of embodiment are perceived as human-like across many contexts, while expressions of morality contribute to anthropomorphism more in emotionally charged situations. Our taxonomy is useful for identifying and measuring language that might contribute to anthropomorphism by assessing the incidence of a particular category of expressions in some language technology output as well as for developing hypotheses like these about anthropomorphism. Additionally, researchers can use the taxonomy to study the anthropomorphic effects of multiple types of expressions present together in an output text, as often occurs in the wild: For instance, might expressions of intelligence counteract the anthropomorphic effects of expressions of limitations? Or perhaps simultaneous expressions of intelligence and limitations intensify anthropomorphism?

Conclusion. In this paper, we taxonomized how text outputs from empirical cases of user interactions with language technologies can contribute to anthropomorphism. In doing so, our taxonomy offers a shared vocabulary and further conceptual clarity for more precise discussions about anthropomorphism of language technologies. We encourage researchers and technologists to use our taxonomy for more targeted identification and mitigation of harmful impacts stemming from anthropomorphism of language technologies.

Limitations. In this work, we use empirical cases of anthropomorphism of language technologies, informed by existing research on anthropomorphic expressions, to develop a taxonomy of textual expressions that can contribute to anthropomorphism of language technologies. Notably, in order for us to collect them, our cases had to be shared publicly. Though we did achieve coverage of prior work, plus some, our sample of public, often high-profile cases may not comprehensively cover all existing text expressions that can contribute to anthropomorphism of language technologies. While we examined text outputs, not how those outputs were generated, it is important to note that the hype around generative AI can influence the extent to which people anthropomorphize text outputs. Additionally, most of our cases ultimately came from LLM-based systems, for which people might have particular expectations and mental models, and with which people interact with in particular ways, such as using dialogue; this might have shaped what was identified as human-like and thus included in our case collection. Annotations of output texts were conducted in English by our research team that is used to writing and reading in standard American English and thus likely biased to have that as a norm when thinking about language and language use.