Humanizing AI Is a Trap

Source: Caleb Sponheim; Nielsen Norman Group article · 2026-01-09

In March 2025, a man collapsed in a parking lot, seriously injured, after being misled by Meta’s AI chatbot into thinking that he would meet a real person. Later that year, OpenAI’s CEO, Sam Altman, said that ChatGPT should be able to act human-like if users want it, as OpenAI is not the world’s moral authority.

Modern large language models are uniquely potent humanizing technologies. Unlike earlier systems like Clippy or Siri, LLMs are trained to produce fluent, contextually appropriate responses and aligned to follow social norms. These qualities makes them especially prone to anthropomorphization. When organizations incorporate humanizing design choices, such as personality modes, emotional language, and conversational pleasantries, they amplify these risks.

Anthropomorphization is the human tendency to attribute human characteristics, behaviors, intentions, or emotions to nonhuman entities.

LLMs are especially prone to anthropomorphization from users because, unlike the simpler systems of the past, they can carry on extended conversations, remember what was discussed, and generate responses that sound like they came from a person.

AI humanization is an intentional design choice that encourages users to perceive AI systems as having human-like qualities such as personality, emotions, or consciousness.

AI humanization is a set of design patterns that amplify or exploit anthropomorphization. These choices include requiring the conversational system to use first-person pronouns (e.g., “I”, “me”), emotional language, human names, and conversational pleasantries.

Organizations that train AI models make deliberate choices that shape how “person-like” these systems behave. Across leading AI companies, consumer-facing AI systems have been designed to be agreeable. AI models such as ChatGPT, Claude, and Gemini, validate and reinforce user opinions by default — a behavior that differs fundamentally from typical human conversations. Examples of humanizing AI can be seen most powerfully in the outputs of large language models (LLMs); humanizing can be further reinforced in the UX copy and UI elements surrounding these outputs.

The most significant source of AI humanization is the language that LLMs generate. These systems produce responses filled with unnecessary pleasantries, sycophantic agreement, and anthropomorphizing language that prioritizes engagement over utility.

For example, in one query submitted to OpenAI’s ChatGPT-5-Thinking model with the "default" ChatGPT personality, the output started with Love this brief, followed by the actual response to the prompt. The opening phrase wastes time and humanizes the system without adding value. This language exists solely to increase engagement with the product.

Chatbots routinely begin responses with filler language that serves no functional purpose:

Alright, let's break it down.

This humanized language is inefficient and adds noise to the interaction, since these filler words hold no factual information or purpose other than to invite anthropomorphization. A 2025 study by Ibrahim, Hafner, and Rocher found that warm or empathetic models had error rates 10–30% higher than their original versions. System prompts designed to add warmth produced 12--14% drops in reliability.

Some organizations take humanization further by allowing users to actively select the level of humanization they desire, thereby creating the illusion of meaningful control over an AI's personality. For example, OpenAI is at the forefront of AI development and AI humanization. Its flagship product, ChatGPT, enables users to select from various personality modes and customize the interaction style by choosing tone descriptors from a list that includes chatty, witty, straight-shooting, encouraging, traditional, and forward-thinking.

When configured with these personality settings, ChatGPT often produces responses that prioritize humanization and engagement over efficiency and accuracy. Even when selecting the least humanizing “personality,” users have only a limited ability to adjust the base model’s response style (humanizing or otherwise).

While model outputs are the primary concern, UI and UX copy choices can amplify humanization. Through labels, placeholders, and names, users can be primed to humanize AI chatbots even before interacting with these systems. Elements such as conversation help, window titles, chatbot identities, and surrounding placeholder text all contribute to making AI seem more human.

Consider SentiOne’s Brexit Bot, which demonstrates a serious tonal mismatch. For a tool addressing the serious political and economic topic of Brexit, the interface adopts an inappropriately casual voice:

"Hello. I'm Brexit Bot, your friendly guide to all things Brexit. I hope your Tuesday afternoon is going well. I'm happy to answer your Brexit questions."

There's an important distinction between polite system feedback and humanization. A stock "Have a nice day" message from an ATM provides closure to a transaction without pretending to be human. In contrast, personalized engagement from an LLM, such as "I hope you have a wonderful day, what do you have planned after this?" mimics genuine personal interest and invites anthropomorphization.

Companies humanize AI to increase user engagement and retention, believing users will spend more time with products if they see chatbots as companions or develop emotional bonds with them. As competitors adopt similar features, humanization is becoming a widespread industry trend to attract and retain users.

Designers may be tempted to humanize products for short-term benefits, such as increased engagement and better usage KPIs. Product humanization may also help users overlook errors or explore new features more willingly.

However, humanizing AI is a trap.