Generator-Retriever-Generator: A Novel Approach to Open-domain Question Answering

Open-domain question answering (QA) tasks usually require the retrieval of relevant information from a large corpus to generate accurate answers. We propose a novel approach called Generator-Retriever-Generator (GRG) that combines document retrieval techniques with a large language model (LLM), by first prompting the model to generate contextual documents based on a given question. In parallel, a dual-encoder network retrieves documents that are relevant to the question from an external corpus. The generated and retrieved documents are then passed to the second LLM, which generates the final answer. By combining document retrieval and LLM generation, our approach addresses the challenges of opendomain QA, such as generating informative and contextually relevant answers. GRG outperforms the state-of-the-art generate-then-read and retrieve-then-read pipelines (GENREAD and RFiD) improving their performance at least by +5.2, +4.2, and +1.6 on TriviaQA, NQ, and WebQ datasets, respectively. We provide code, datasets, and checkpoints.1.

Introduction. Open-domain question answering (QA) tasks pose significant challenges since they require access to large document collections or repository of domain knowledge. Existing methods for QA (Chen et al., 2017; Karpukhin et al., 2020; Izacard and Grave, 2020) often rely on a retrieve-then-read pipeline, where relevant contextual documents are retrieved from external sources like Wikipedia, and the answer prediction is conditioned on these documents and the question. These methods suffer however from several drawbacks. Firstly, the retrieved documents are often chunked and fixed, which can result in the inclusion of noisy and irrelevant information. The fixed-size document chunks may not adequately capture the context necessary for finding accurate answers (Yu et al., 2022; Abdallah et al., 2020). Consequently, the presence of irrelevant information can lead to noise in the retrieved documents, negatively impacting the quality and relevance of the generated answers.

Discussion / Conclusion. In this paper, we proposed a Generator-Retriever- Generator approach for improving open-domain question answering systems. By combining generated and retrieved documents, we achieved significant performance gains across multiple benchmark datasets. Our experiments demonstrated that GRG outperformed existing baselines in terms of accuracy and efficiency. The results indicate the effectiveness of incorporating both generated and retrieved documents in the reading process, leveraging the combined strengths of language models and retrieval systems.Future work should focus on improving the accuracy of the document retrieval approach, potentially through the use of more advanced retrieval models or incorporating additional contextual information. Further investigations into hyperparameter configurations, such as the number of generated and retrieved documents. 1. The accuracy and relevance of generated documents impact the overall performance. As with any language model, there is a possibility of generating inaccurate or irrelevant information.