Learning to Relate to Previous Turns in Conversational Search

As in any conversation in natural language, queries in conversational search may involve omissions, references to previous turns, and ambiguities [32]. Thus, a primary challenge for effective conversation search is to determine the underlying information need by understanding context-dependent query turns. Some previous studies [17, 37, 40, 41, 43] exploited a two-stage pipeline, which first trains a query rewriting (QR) model with external data or human reformulated queries, then feeds the rewritten queries into a retriever. However, it is required that sufficient manual annotations are available for model training, which is difficult to obtain in practice [1, 2]. Furthermore, the manually reformulated queries may not be the best search queries because the reformulation is based on human understanding of the query rather than on the search results. We also observe that the rewriting model is usually trained separately from the retriever, making it difficult to optimize the whole system for the final retrieval performance [20, 41]. In particular, the learned rewriting model may not best fit the retriever used in the subsequent step.

Another research direction [15, 24, 30, 44] tries to train a conversational query encoder by leveraging all the previous context information. A common approach uses all the historical queries in the session to expand the current query. Although this expansion leads to improved results, it makes a strong assumption that all the previous queries are related to the current one. In reality, we often observe topic switches within a conversation session, which means the current query may not be related to the previous ones. Fig. 1 illustrates such an example, where we see that only part of the historical queries is relevant to the last query (either by human annotators or by our model). Simply leveraging all the historical queries will inevitably inject irrelevant information into the expanded query and result in sub-optimal queries.

“Conversational search allows a user to interact with a search system in multiple turns. A query is strongly dependent on the conversation context. An effective way to improve retrieval effectiveness is to expand the current query with historical queries. However, not all the previous queries are related to, and useful for expanding the current query.

....

So the crucial question is: how can we select useful information from history context for query expansion/reformulation?

...we investigate how to select useful historical turns to expand the current query in conversational search. The selection is made according to the usefulness of the previous queries, i.e. if the latter can help improve the retrieval effectiveness of the current query. Intuitively, this could help select more useful previous queries for the search task.

1. We propose an efficient and effective method to select useful previous queries to expand the current query without human annotations.

(2) We propose a multi-task learning method to jointly optimize the query selection model and the dense retrieval model.

(3) We demonstrate the effectiveness of the proposed method and show its broad applicability in various settings (i.e. with or without training data). In addition, the proposed method can select better previous queries than human annotators.”