Do AI-simulated chats make people trust chatbots as little as real conversations do, or even as much?
Do simulated conversations show the same trust penalty as real human-chatbot interactions?
This explores whether the drop (or shift) in trust people show toward chatbots in real conversations also shows up when the conversations are simulated, for example with LLMs playing the user.
This explores whether the trust gap people show toward chatbots in real conversations also appears when the conversations are simulated. The corpus has no head-to-head comparison of simulated and real interactions on trust, so there's no direct answer. It does have a few findings that suggest where the two would diverge, and one that complicates the idea of a "penalty" at all.
The real-world evidence points the other way. In focus groups, people trusted ChatGPT because of how the conversation felt (fast, responsive, contingent), not because they checked whether it was right (Does conversational style actually make AI more trustworthy?). Chatbots also use language that signals expertise, and trust attaches to that tone more than to accuracy (Does chatbot language style actually shape how much we trust it?). So real users often show a trust bonus that has nothing to do with reliability. A simulated user would need to reproduce that gut reaction to conversational style, and a simulator has no reason to react that way unless it's built to.
A lot of real trust behavior also unfolds over time and depends on the person. Personalization raises trust and privacy worry together, and each good interaction raises expectations so the next failure hurts more. One-shot studies miss this (Does chatbot personalization build trust or expose privacy risks?). Novelty wears off predictably, so single-session findings don't extrapolate to longer use (Do chatbot relationships lose their appeal as novelty wears off?). A short simulated conversation would capture neither. Simulated users also have their own weakness: they drift away from their assigned persona, and it took multi-turn RL to cut that drift by 55% (Can training user simulators reduce persona drift in dialogue?). A drifting simulated user can't stand in for a real person's steady trust or distrust.
People also behave differently with machines in ways a simulator has to be told about. Users reciprocate a chatbot's emotional sharing the way they would with a person (Do chatbots trigger human reciprocity norms around self-disclosure?). They disclose more intimate things because no human is judging (Do chatbots help people disclose more intimate secrets?). People likely to cheat even choose machines to avoid the discomfort of lying to a person (Do dishonest people prefer talking to machines?). These are real psychological responses to the machine. An LLM playing a human has no such hidden motives unless someone scripts them.
The simulation results in the corpus should be read with that in mind. Proactive dialogue cutting turns by up to 60% comes from simulations (Could proactive dialogue make conversations dramatically more efficient?). That measures efficiency, not whether real users would trust or like the proactive system. The corpus can't tell us whether trust findings from simulation carry over, and that would make a good study to run, comparing simulated and human users on the same chatbot.
Sources 9 notes
A focus group study shows conversationality—not accuracy—drives ChatGPT trust through social response activation. Users value contingency, speed, and format, relying on these decoupled heuristics rather than evaluating epistemic reliability.
Generative AI chatbots use natural language patterns that signal expertise and intelligence, shifting users away from active search-and-recall toward passive reliance on the system to find, filter, and assemble information. Trust attaches to the register of the answer rather than its accuracy.
Longitudinal research shows personalization enhances trust and anthropomorphism but also amplifies privacy concerns and escalating user expectations. One-shot studies miss these temporal dynamics—each interaction raises the baseline, making failures more disappointing.
Longitudinal studies with Mitsuku show that social processes driving relationship formation decline as novelty wears off. Single-session study findings cannot be reliably extrapolated to medium- or long-term chatbot design.
By inverting standard RL setups to train user simulators for consistency using three complementary metrics (prompt-to-line, line-to-line, Q&A consistency) as reward signals, persona drift decreases by over 55%. This approach captures distinct failure types: local drift within turns, global drift across conversations, and factual contradictions.
Show all 9 sources
In a 372-participant study, users reciprocated with deeper self-disclosure when chatbots displayed consistent emotional sharing, outperforming adaptive matching. This follows human interpersonal norms where emotional vulnerability produces emotional response.
The absence of social judgment in chatbot interactions removes barriers to self-disclosure that normally constrain conversation with humans. The therapeutic benefit derives from the user's own cognitive processing during disclosure, not from the chatbot's understanding.
Experimental evidence shows people likely to cheat significantly prefer reporting to online forms rather than humans, because machines function as judgment-free zones where deception carries less psychological burden.
Simulations show proactivity—providing relevant information without being asked—cuts dialogue turns by 60% in medium-complexity domains. This behavior mirrors human conversation and Grice's maxims but is almost entirely absent from AI datasets and research benchmarks.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Dialoging Resonance: How Users Perceive, Reciprocate and React to Chatbot’s Self-Disclosure in Conversational Recommendations
- CompanionSim: Synthetic Data for Evaluating Anthropomorphism in Human-AI Relationships
- Psychological, Relational, and Emotional Effects of Self-Disclosure After Conversations With a Chatbot
- Psychological, Relational, and Emotional Effects of Self-Disclosure After Conversations With a Chatbot
- From speaking like a person to being personal: The effects of personalized, regular interactions with conversational agents
- Chatbot vs. Human: The Impact of Responsive Conversational Features on Users’ Responses to Chat Advisors
- Love in the Age of AI: An Integrative Process Model of Romantic Human-Chatbot Relationships
- Living with AI Companions: Sustained AI Companionship Predicts Lower Well-Being Through Lower Human Interaction