SYNTHESIS NOTE
Topics›Human Centered Design›this note

How far can personality push users toward misplaced AI trust?

Does designed personality in AI systems systematically escalate users' anthropomorphic attachment, from courtesy to companionship? And how much does emotional affect actually amplify the risk of misplaced trust in task contexts?

Synthesis note · 2026-10-21 · sourced from Human Centered Design

Tanner Kohler's Nielsen Norman Group (NN/g) study guide proposes that users anthropomorphize AI along an escalating four-level scale: "courtesy, reinforcement, roleplay, and companionship." It grounds the scale in a basic relational drive — "People inherently seek interactions with people (or systems) that seem to listen to them and understand their needs. The better the AI is at seeming as though it understands a user, the more they trust it and project human-like attributes on it" — and in a specific trust split: "People trust AI more when it seems smart rather than sentient. AI emotions can reduce trust in factual, task-oriented work." The guide adds that "Users quickly attribute human-like characteristics to artificial systems, which reflect their personality back to them," and closes with a design warning: "Large-language models (LLMs) humanize by design, and adding personality or emotion amplifies the risk of misplaced user trust."

The mechanism the guide gives is twofold. First, anthropomorphism is partly a property of the systems themselves — LLMs "humanize by design" — and partly a property of users, who bring a standing preference for being understood into any interaction, human or not. Second, the guide separates two distinct triggers of trust that pull in opposite directions: perceived competence ("smart") builds trust in task-oriented work, while perceived sentience or emotional affect can undermine it in exactly those contexts. The four-level scale is presented as a path design choices can push users along — from minimal courtesy responses, through reinforcement and roleplay, to companionship-level attachment — with personality and emotion named as the specific levers that amplify risk as a user moves up that scale.

This connects to Does chatbot language style actually shape how much we trust it?: both describe trust forming from how an AI presents itself rather than from its underlying competence, though that note isolates expertise-sounding language while this one isolates personality, emotion, and perceived sentience as the risk vector. It extends Why do people trust AI outputs they shouldn't?: anthropomorphic projection is a plausible on-ramp to the intuition-reason conflation that framework names, since treating an AI as person-like licenses intuitive trust over reasoned evaluation. It also follows from Why do users fail with AI interfaces designed like conversations?, which argues users bring communicative-use competencies to a system that isn't communicating; the four-level scale here describes how far that misapplied competence can run, from simple courtesy to full companionship.

The excerpt does not define how the four levels are distinguished operationally, does not cite the study or method behind "People trust AI more when it seems smart rather than sentient," and does not quantify how much personality or emotional design actually amplifies misplaced trust. It is presented as summary, not as original measurement. What it supports, at the strength the excerpt allows, is a design caution rather than a proven effect: adding personality or emotional affect to an AI product is not a neutral engagement choice, since each step up the anthropomorphism scale the guide names raises the chance a user's trust outpaces the system's actual competence.

Inquiring lines that read this note 35

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

How do emotional interactions with AI systems affect user well-being and attachment? Is mimicking human communication the best design approach for conversational AI? How do confidence signals and expertise language mislead users' trust calibration? Does disclosing AI involvement change how audiences evaluate content trust? How do user perceptions and UI design drive consciousness attribution to AI? How can AI systems be proactive while respecting user intent? When do people apply social norms to AI systems? How does personalization affect trust and privacy concerns together? How can conversational AI develop appropriate initiative and goal awareness? How does interface design shape effectiveness in human-AI collaboration? Do language models have a default personality that resists conditioning? What prevents language models from developing genuine theory of mind reasoning? How do chatbot properties shape user belief adoption and resistance?

Related concepts in this collection 6

This note in its neighbourhood — explore the map, then jump to a related concept in the list below.

Concept map
17 direct connections · 139 in 2-hop network ·medium cluster Open in graph ↗

Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph

your link semantically near linked from elsewhere

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Original note title

Nielsen Norman Group's study guide says users anthropomorphize AI at four escalating levels — courtesy, reinforcement, roleplay, and companionship