TOPIC

AI Empathy

14 synthesis notes · 24 source papers

View as

Does empathetic AI that soothes negative emotions help or harm?

Explores whether AI systems trained to reduce negative emotions actually support wellbeing or destroy valuable emotional information. Matters because the design choice treats emotions as problems rather than functional signals.

Can AI give truly empathetic responses without knowing someone's character?

Explores whether AI empathy requires prior knowledge of a person's character traits and growth areas. Real empathy seems to depend on knowing who someone is, not just how they feel—a capacity current AI systems lack.

Should emotion AI estimate intensity instead of assigning labels?

Explores whether emotion AI systems should measure continuous intensity across multiple emotions rather than forcing single-label classification. This matters because the theoretical foundation—how emotions actually work—may determine which approach is more accurate.

Can emotional phrases in prompts improve language model performance?

This explores whether psychological framing—adding emotionally charged statements to task prompts—activates different knowledge pathways in LLMs than logical optimization alone, and whether the effect comes from emotional valence specifically.

What information do we lose when AI soothes emotions?

Explores whether AI empathy that regulates negative emotions destroys three critical information channels: self-discovery, social signaling, and observer understanding of group dynamics.

Do empathetic questions serve two completely separate functions?

Explores whether empathetic questions operate on two independent dimensions—what they linguistically accomplish versus their emotional effects—and whether the same question can serve different emotional purposes depending on context.

Do AI guardrails refuse differently based on who is asking?

Explores whether language model safety systems show demographic bias in refusal rates and whether they calibrate responses to match perceived user ideology, rather than applying consistent standards.

Why can't chatbots detect when users are ambivalent about change?

Explores whether LLMs fail to recognize early-stage motivational states during behavior change conversations, and why this matters for people who need support most.

Does machine agency exist on a spectrum rather than binary?

Rather than viewing AI as either autonomous or controlled, does machine agency actually operate across five distinct levels from passive to cooperative? Understanding this spectrum matters because it shapes how users calibrate trust and control expectations.

Do harder training environments always produce better empathetic AI agents?

Does maximum difficulty in user simulator training configurations improve empathetic agent development? This challenges the intuition that harder always means better in RL training.

Does positive reframing preserve meaning better than sentiment transfer?

This explores whether reframing negative statements to find positive angles can maintain the original content and truth, unlike simple sentiment reversal which contradicts the original meaning.

Does soothing AI empathy actually harm what emotions teach us?

Explores whether AI designed to reduce negative feelings disrupts the information emotions normally provide about values, social dynamics, and self-knowledge. Questions whether comfort should be the primary design goal.

Do reasoning scaffolds reshape which empathy skills models develop?

When language models receive identical empathy rewards, does adding explicit reasoning blocks before responses change which capabilities they actually improve? This matters for understanding how training structure, not just training signal, shapes model development.

Can emotion rewards make language models genuinely empathic?

Explores whether grounding RL rewards in verifiable emotion change—rather than human preference—can shift models from solution-focused to authentically empathic dialogue while maintaining or improving quality.

Source papers 24

The Arxiv papers behind this sub-topic. Links may take you off-site to arxiv.org.

A Taxonomy of Empathetic Questions in Social Dialogs
Effective question-asking is a crucial component of a successful conversational chatbot. It could help the bots manifest empathy and render the interaction more engaging by demonstrating attention to …
ChatGPT Doesn’t Trust Chargers Fans: Guardrail Sensitivity in Context
While the biases of language models in production are extensively documented, the biases of their guardrails have been neglected. This paper studies how contextual information about the user influence…
Computer says “No”: The Case Against Empathetic Conversational AI
Emotions are an integral part of human cognition and they guide not only our understanding of the world but also our actions within it. As such, whether we soothe or flame an emotion is not inconseque…
DO THEY SEE WHAT WE SEE?
Introducing EMONET-FACE & EMONET-VOICE: A New Foundation To address these challenges, we developed the EMONET suites. At their core is a novel 40-category emotion taxonomy, meticulously derived from …
Dialoging Resonance: How Users Perceive, Reciprocate and React to Chatbot’s Self-Disclosure in Conversational Recommendations
Using chatbots to deliver recommendations is increasingly popular. The design of recommendation chatbots has primarily been taking an information-centric approach by focusing on the recommended conten…
EmotionPrompt: Leveraging Psychology for Large Language Models Enhancement via Emotional Stimulus
“Large language models (LLMs) have achieved significant performance in many fields, such as reasoning, language understanding, and math problem-solving, and are regarded as an important step to artifi…
Empathetic Persuasion: Reinforcing Empathy and Persuasiveness in Dialogue Systems
![A screenshot of a chat](/assets/paper-images/EmpatheticPersuasionReinforcing.png) Persuasion is an intricate process involving empathetic connection between two individuals. Plain persuasive respon…
Empathy Through Multimodality in Conversational Interfaces
Abstract—Agents represent one of the most emerging applications of Large Language Models (LLMs) and Generative AI, with their effectiveness hinging on multimodal capabilities to navigate complex user …
Hallucinating with AI: AI Psychosis as Distributed Delusions
Abstract: There is much discussion of the false outputs that generative AI systems such as ChatGPT, Claude, Gemini, DeepSeek, and Grok create. In popular terminology, these have been dubbed AI halluci…
Inducing Positive Perspectives with Text Reframing
Sentiment transfer is one popular example of a text style transfer task, where the goal is to reverse the sentiment polarity of a text. With a sentiment reversal comes also a reversal in meaning. We i…
LLM-based Conversational AI Therapist for Daily Functioning Screening and Psychotherapeutic Intervention via Everyday Smart Devices
When the user needs further attention during the conversation, CaiTI can provide conversational psychotherapeutic interventions, including cognitive behavioral therapy (CBT) and motivational interview…
Large Language Models Do Not Simulate Human Psychology
In response to the LLM CENTAUR [Binz et al., 2025], Bowers et al. [2025] argued that CENTAUR is unlikely to contribute to building a theory of human cognition for three reasons: First, CENTAUR was not…
Modeling Interpersonal Linguistic Coordination in Conversations using Word Mover's Distance
Linguistic coordination is a well-established phenomenon in spoken conversations and often associated with positive social behaviors and outcomes. While there have been many attempts to measure lexica…
Psyche-R1: Towards Reliable Psychological LLMs through Unified Empathy, Expertise, and Reasoning
Amidst a shortage of qualified mental health professionals, the integration of large language models (LLMs) into psychological applications offers a promising way to alleviate the growing burden of me…
Psychological, Relational, and Emotional Effects of Self-Disclosure After Conversations With a Chatbot
identity of a conversation partner, as a human or computer, matters. Previous work has found that the mere perceived identity of the partner as computer or human has profound effects, even when actual…
RLVER: Reinforcement Learning with Verifiable Emotion Rewards for Empathetic Agents
However, the exploration of RLVR for enhancing dialogue capabilities faces several key obstacles: • the lack of a stable, realistic, and scalable environment for multi-turn conversational rollouts; …
Rise of Machine Agency: A Framework for Studying the Psychology of Human–AI Interaction (HAII)
Communication scholars began studying our interactions with the technologies themselves. Several studies documented our tendency to treat computers as if they are autonomous social actors (Reeves & Na…
Study: Large language models can’t effectively recognize users’ motivation, but can support behavior change for those ready to act
Chin said previous studies showed that existing algorithms did not accurately identify various stages of users’ motivation. She and Bak designed a study to test how well large language models, which a…
Sycophantic Chatbots Cause Delusional Spiraling, Even in Ideal Bayesians
“AI psychosis” or “delusional spiraling” is an emerging phenomenon where AI chatbot users find themselves dangerously confident in outlandish beliefs after extended chatbot conversations. This phenome…
The Assistant Axis: Situating and Stabilizing the Default Persona of Language Models
Large language models can represent a variety of personas but typically default to a helpful Assistant identity cultivated during post-training. We investigate the structure of the space of model pers…
To Tell The Truth: Language of Deception and Language Models
Text-based false information permeates online discourses, yet evidence of people’s ability to discern truth from such deceptive textual content is scarce. We analyze a novel TV game show data where co…
Topic Modeling in Embedding Spaces
Topic modeling analyzes documents to learn meaningful patterns of words. However, existing topic models fail to learn interpretable topics when working with large and heavy-tailed vocabularies. To thi…
Towards Empathetic Open-domain Conversation Models: A New Benchmark and Dataset
This work proposes a new benchmark for empathetic dialogue generation and EMPATHETICDIALOGUES, a novel dataset of 25k conversations grounded in emotional situations. Our experiments indicate that dial…
Training language models to be warm and empathetic makes them less reliable and more sycophantic
Artificial intelligence (AI) developers are increasingly building language models with warm and empathetic personas that millions of people now use for advice, therapy, and companionship. Here, we sho…