Empathetic Persuasion: Reinforcing Empathy and Persuasiveness in Dialogue Systems

Persuasion is an intricate process involving empathetic connection between two individuals. Plain persuasive responses may make a conversation non-engaging. Even the most wellintended and reasoned persuasive conversations can fall through in the absence of empathetic connection between the speaker and listener. In this paper, we propose a novel task of incorporating empathy when generating persuasive responses. We develop an empathetic persuasive dialogue system by fine-tuning a Maximum Likelihood Estimation (MLE)-based language model in a Reinforcement Learning (RL) framework. To design feedbacks for our RLagent, we define an effective and efficient reward function considering consistency, repetitiveness, emotion and persuasion rewards to ensure consistency, non-repetitiveness, empathy and persuasiveness in the generated responses. Due to lack of emotion annotated persuasive data, we first annotate the existing PERSUAION- FORGOOD dataset with emotions, then build transformer based classifiers to provide emotion based feedbacks to our RL agent.

Introduction. While conversing with persuasive dialogue agents, on top of fluent and meaningful response generation, a high quality conversation is often derived by understanding and acknowledging implied feelings towards the conversing partner. People are more likely to engage in the conversation when they are motivated with empathetic responses. These persuasive responses can be associated with differ- ent emotions in consonance with the way people perceive and think about the world. For instance, in Figure 1, while the strike-through response is persuasive, the green box response may be more engaging, as it connects with the end-user and acknowledges the underlying emotion of caring. In this work, we investigate different generic and task specific rewards to reinforce a dialogue agent to generate fluent, persuasive and empathetic responses.

Discussion / Conclusion. Development of persuasive dialogue agents to generate empathetic responses is still in its nascent stage due to the lack of modelling the changing attitudes of individuals. Further, generative models only with MLE loss may lead to exposure bias and tend to generate generic responses. Therefore, to connect with end-users empathetically and generate goal oriented-responses, we propose here an RL-based dialogue generation framework adopting PPO method to fine-tune the model. To force the agent to generate more empathetic and persuasive responses, we define an efficient and effective reward function considering two generic rewards, viz. consistency and repetitiveness and two taskspecific rewards i.e. emotion reward - which forces the agent towards empathetic responses and persuasive reward - which forces the agent to generate persuasive responses. Automatic and human evaluation results demonstrate that by just adding extra reward of emotion, our model is able to achieve state-of-the-art result in a complex task like persuasion, and generate consistent, non-repetitive, empathetic and persuasive responses 4.