HiTKG: Towards Goal-Oriented Conversations via Multi-Hierarchy Learning

Human conversations are guided by short-term and longterm goals. We study how to plan short-term goal sequences as coherently as humans do and naturally direct them to an assigned long-term goal in open-domain conversations. Goal sequences are a series of knowledge graph (KG) entityrelation connections generated by KG walkers that traverse through the KG. The existing recurrent and graph attention based KG walkers either insufficiently utilize the conversation states or lack global guidance. In our work, a hierarchical model learns goal planning in a hierarchical learning framework. We present HiTKG, a hierarchical transformer-based graph walker that leverages multiscale inputs to make precise and flexible predictions on KG paths. Furthermore, we propose a two-hierarchy learning framework that employs two stages to learn both turn-level (short-term) and global-level (long-term) conversation goals. Specifically, at the first stage, HiTKG is trained in a supervised fashion to learn how to plan turn-level goal sequences; at the second stage, HiTKG tries to naturally approach the assigned global goal via reinforcement learning. In addition, we propose MetaPath as the backbone method for KG path representation to exploit the entity and relation information concurrently.

Introduction. Building a human-like dialogue system has been a longlasting goal in the community of conversational AI (Ni et al. 2021; Ma et al. 2020). In the pursuit of this goal, multiple research topics have emerged: context awareness (Qiu et al. 2020), response coherence (Liu et al. 2019a) and diversity (Su et al. 2020), speaker consistency (Madotto et al. 2019), empathetic response (Song et al. 2019), conversation topic (Wu et al. 2019), knowledge-grounded system (Chen et al. 2020), etc. Conversation goal is one of the most representative elements that reflect human intelligence. Human conversations are usually guided by several small goals or a global goal. As shown in Fig. 1, Grilled Fish, Chinese Dish, China Town, and Cinema are turn-level goals, while the Cinema is also the global goal at the same time. During the conversation, the agent intends to approach the global goal by naturally transitioning between turn-level goals. However, most dialogue systems passively respond to the user without explicit goals, causing incoherent or illogical responses.

Discussion / Conclusion. We propose HiTKG, a hierarchical transformer based KG walker that leverages multiscale inputs for graph reasoning in dialogues. HiTKG first learns to plan natural turn-level goals and then learns to approach a global goal. Both automatic and human evaluation illustrate the effectiveness of our method. In the future, we will investigate how to improve the embedding, learning framework, and evaluation criteria of stage 2 to further extend this topic.