Improving Dialog Systems for Negotiation with Personality Modeling

In this paper, we explore the ability to model and infer personality types of opponents, predict their responses, and use this information to adapt a dialog agent’s high-level strategy in negotiation tasks. Inspired by the idea of incorporating a theory of mind (ToM) into machines, we introduce a probabilistic formulation to encapsulate the opponent’s personality type during both learning and inference. We test our approach on the CRAIGSLISTBAR- GAIN dataset (He et al., 2018) and show that our method using ToM inference achieves a 20% higher dialog agreement rate compared to baselines on a mixed population of opponents. We also find that our model displays diverse negotiation behavior with different types of opponents.1

Introduction. Developing dialog systems for negotiation is challenging since the task requires a combination of good communication skills and strategic reasoning capabilities (Traum et al., 2008; Young et al., 2013; Keizer et al., 2017). While recent neural models (Wen et al., 2017; Dhingra et al., 2017; Zhou et al., 2019; He et al., 2018) have shown that useful dialogue strategies can be learned from offline corpora, they do not explicitly model the mental state of other agents, which can make it challenging to generate tailored strategies and utterances for different types of opponents. In this paper, we introduce a new framework for generating strategic dialog inspired by the idea of Theory of Mind (ToM) from cognitive science (Premack and Woodruff, 1978; Bruner, 1981; Wimmer and Perner, 1983). When negotiating with others, humans innately infer the intention of the other party, and guess how their own utterances would affect the opponent’s mental state.

Discussion / Conclusion. In this work, we proposed a novel framework to integrate the concept of Theory of Mind (ToM) into generating task-oriented dialogs. Our approach provides the ability to model and infer personality types of opponents, predict changes in their mental state, and use this information to adapt the agent’s high-level strategy in negotiation tasks. We in- troduced a probabilistic formulation for first-order ToM and introduce two ways to incorporate it into a dialog agent, by 1) explicitly and 2) implicitly modeling the personality of the opponent. We tested our approach on a modified version of the CRAIGSLIST- BARGAIN dataset (He et al., 2018) with diverse opponents. Our experiments show that our method using ToM inference achieves about 20% higher dialog agreement rate and utility compared to baselines on a mixed population of opponents. When negotiating with the cooperative opponents, the improvement of agreement rate is 54%. Some directions for future work include developing efficient schemes to approximate the value computation for future states, exploring higher orders of ToM, as well as a tighter integration of ToM into utterance generation and processing.