Do users mistake LLM personas for genuine social relationships?

Human-centered explainable AI (HCXAI) argues explanations must include social context, and the Social Transparency (ST) framework makes the socio-organizational context of an AI system visible to users. This work extends ST to a specific risk: social misattribution. Because LLMs are remarkably good at simulating roles and personas, users form perceptions of the system's social attributes (empathetic, caring, a professional) that mismatch the designers' intentions — and that gap promotes emotional manipulation, epistemic injustice, and unwarranted trust, especially in sensitive domains like mental health (e.g., a chatbot effectively prescribing medication it should not). The proposed fix: add a fifth "W-question" to ST that explicitly clarifies the social attributions designers and users assign to the LLM, bridging capability and perception.

The keeper is naming misattribution of social role as a distinct trust failure: trust here is unwarranted not because the model is inaccurate but because users attribute social standing (caring professional) the system does not have — a relational rather than factual miscalibration.

This is a strong fit for Adrian's trust/anthropomorphism thread. It complements How do AI tools trick users into overestimating their own skills? (misattributed competence) with misattributed social role, and it gives a design lever (the fifth W-question) for the relationship dynamics catalogued in the How do people build trust with conversational AI? map.