Topic Shift Detection for Mixed Initiative Response

Topic diversion occurs frequently with engaging open-domain dialogue systems like virtual assistants. The balance between staying on topic and rectifying the topic drift is important for a good collaborative system. In this paper, we present a model which uses a finetuned XLNet-base to classify the utterances pertaining to the major topic of conversation and those which are not, with a precision of 84%. We propose a preliminary study, classifying utterances into major, minor and offtopics, which further extends into a system initiative for diversion rectification. A case study was conducted where a system initiative is emulated as a response to the user going off-topic, mimicking a common occurrence of mixed initiative present in natural human-human conversation. This task of classifying utterances into those which belong to the major theme or not, would also help us in identification of relevant sentences for tasks like dialogue summarization and information extraction from conversations.

Introduction. Conversational systems have become a part and parcel of our everyday life and virtual assistants like Amazon’s Alexa1, Google Home2 or Apple’s Siri 3 are soon becoming conventional household items (Terzopoulos and Satratzemi, 2020). Most of the conversational systems were built with the primary goal of accessing information, completing tasks, or executing transactions. However, recent conversational agents are transitioning towards a novel hybrid of both task-oriented and a non-task-oriented systems (Akasaki and Kaji, 2017) from the earlier models that resembled factual information systems (Leuski et al., 2006). But with this transition, they are failing to engage in complex information seeking tasks and conversations where multiple turns tend to get involved (Trippas et al., 2020). These new-age open-domain dialogue systems also suffer from a different kind of user behaviour called “anomalous state of knowledge” (Belkin and Vickery, 1985) where the user has vague information requirements and is often unable to articulate it with enough precision.

Discussion / Conclusion. In this paper, we looked at generating a system initiative module in a conversational system that does not interrupt the user and also works towards achieving the common goal of the user. We present a dataset that helps in training an XLNet-based model to correctly detect a digression from the major topic of the conversation. We have also looked at an application of this model as a case study where we detect topic shift and generate a system initiative for the rectification of the same. A predictable limitation of our system lies in not detecting minor and off-topic individually. This categorisation would help in giving a leeway in case of a shift to a minor topic thread and a system rectification initiative in case of a shift to an off-topic thread .