Learning to Select the Relevant History Turns in Conversational Question Answering

Abstract. The increasing demand for web-based digital assistants has given a rapid rise in the interest of the Information Retrieval (IR) community towards the field of conversational question answering (ConvQA). However, one of the critical aspects of ConvQA is the effective selection of conversational history turns to answer the question at hand. The dependency between relevant history selection and correct answer prediction is an intriguing but under-explored area. The selected relevant context can better guide the system so as to where exactly in the passage to look for an answer. Irrelevant context, on the other hand, brings noise to the system, thereby resulting in a decline in the model’s performance. In this paper, we propose a framework, DHS-ConvQA (Dynamic H istory Selection in Conversational Question Answering), that first generates the context and question entities for all the history turns, which are then pruned on the basis of similarity they share in common with the question at hand. We also propose an attention-based mechanism to re-rank the pruned terms based on their calculated weights of how useful they are in answering the question.

Introduction. The long-standing objective of the IR community has been to design intelligent agents, whether web-based or mobile-based, that can engage in eloquent inter- action with humans iteratively [1,2,3]. The IR community has come closer to the realization of the dream owing to the rapid progress in conversational datasets and pre-trained language models [3]. These advancements have resulted in the birth of the field of conversational question answering (ConvQA). ConvQA provides a simplified but strong setting for conversational search [4] where the user initiates the conversation with a specific information need in mind. The system attempts to find relevant information pertinent to the question at hand iteratively based on a user’s response or follow-up questions [4,5,6]. When answering the follow-up questions, the model needs to take the previous conversational turns into account to comprehend the context [7,8].

Discussion / Conclusion. This paper discusses a significant point of view on the basic concept of the role of relevance in conversational question answering (ConvQA). We argue that many existing research works, even the popular ones, do not take into account the idea of relevant history selection and modeling. We propose a framework that combines the notion of both hard history selection and soft history selection to curate the input for the answer prediction module carefully. The model first generates context and question entities using distant supervision learning and selects the relevant terms using ‘hard history selection’. After the pruning of irrelevant terms, the model assigns attention-based weightage to the remaining turns. The assigned score is based on how relevant they are to the current question and accessed in the same order. To further aid the answering prediction process, we utilize binary classification task to highlight the important terms with respect to the current question from the conversational history. Our experimental results depict that the proposed method has the potential to change how conversational history could be utilized more effectively.