How AI Works and How Users Think About It: Study Guide

Paper · Source
Human-Centered Design

Source: Tanner Kohler; Nielsen Norman Group study guide · 2026-09-25

Let’s not pretend we all completely understand the inner workings of artificial intelligence models. Neither do your users. Thankfully, you don’t need to fully understand them to use them. The same has always been true of a computer.

However, the better you understand some of the basics, the more accurate your mental model is. Accurate mental models make you more efficient and help you avoid problems.

These resources can strengthen your mental model of how AI functions and illuminate the mental models your users have for these systems, which dictate their usage of them.

Neural networks, unsupervised learning, hallucinations, and sycophancy? There is an entire vocabulary around large language models (LLMs) that suddenly gets thrown around like we all know what they are and why they matter. The resources below break these (and other) relevant AI concepts down into normal language.

AI introduces a third UI paradigm called intent-based interaction, where users state goals rather than specific commands.

This is a non-technical explanation of how large-language models (LLMs) process user inputs and create their responses.

Large-language models (LLMs) learn language and develop their individual styles through multiple phases of training.

This article explains why AI tends to hallucinate, why that is hard to change, and how it should affect AI product design decisions.

Large-language models (LLMs) are built to prioritize pleasing users over validating the accuracy of their responses.

An AI agent pursues a goal by iteratively taking actions, evaluating progress, and deciding next steps; useful agents must be reliable, adaptive, and accurate.

With generative UI, the system decides to generate an interactive element or product in response to a user need; with vibe coding, the user asks the AI to build it.

Users often include format or structure specifications in prompts to ensure that the chatbot’s response satisfies those requirements.

Users tend to follow common patterns and structural elements while prompting LLMs.

Research shows six types of AI conversations: search queries, funneling, exploring, chiseling, expanding, and pinpointing.

Two patterns describing the shifting and complex ways in which users carry on task-related conversations with genAI chatbots.

People inherently seek interactions with people (or systems) that seem to listen to them and understand their needs. The better the AI is at seeming as though it understands a user, the more they trust it and project human-like attributes on it.

People trust AI more when it seems smart rather than sentient. AI emotions can reduce trust in factual, task-oriented work.

Users quickly attribute human-like characteristics to artificial systems, which reflect their personality back to them.

It’s easy to place too much trust in genAI tools. Use only information you can verify or recognize to be true.

Users tend to anthropomorphize AI at four different levels: courtesy, reinforcement, roleplay, and companionship.

Large-language models (LLMs) humanize by design, and adding personality or emotion amplifies the risk of misplaced user trust.

LLMs have introduced a new way for users to seek answers to their questions — a role search engines have dominated for years. While AI has made certain aspects of information seeking easier, it is far from perfect. It remains difficult to know whether its answers are true and trustworthy.

AI can help people who don’t know the words for what they are looking for, but many users still don’t use it effectively.

Generative AI is reshaping search, but long-standing habits persist.

Users expect AI chatbots to aggregate information in a concise and specific manner, while fully considering contextual cues.

Users choose AI to explore and synthesize information but rely on traditional search when accuracy and trust are critical.

While there has been an incredible adoption rate of AI across many different user groups, the learning curve can be steep. When users have inaccurate mental models of how the system functions or lack a larger vision of its potential, their usage of it can be underwhelming.

Out of ~5,000 working Americans, 81% reported they did not use AI at work.

Users struggle to understand how complex personalization systems turn their actions into recommendations.

AI literacy requires both prompt fluency and the ability to critically assess outputs; frequent use does not imply skilled use.

Lines of inquiry this paper opens 24

Research framings built by reading the notes related to this paper — the questions it feeds into.

Does personalization in chatbots strengthen engagement or undermine authentic interaction? How do chatbot properties shape user belief adoption and resistance? How do confidence signals and expertise language mislead users' trust calibration? How does personalization affect trust and privacy concerns together? How do emotional interactions with AI systems affect user well-being and attachment? How does AI-generated content affect trust in social conversations? How does AI exposure shape employment demand and worker earnings?