论文标题
开放域质量检查者可以像人类一样有效利用外部知识吗？
Can Open-Domain QA Reader Utilize External Knowledge Efficiently like Humans?
论文作者
论文摘要
最新的最新开放域质量检查模型通常是基于两阶段检索器方法的方法，在该方法中，检索器首先找到相关的知识/段落，然后读者然后利用这些知识/段落来预测答案。先前的工作表明，随着这些段落数量的增加，读者的表现通常会改善。因此，最新模型使用大量段落（例如100）进行推理。尽管这种方法的读者可以实现高预测性能，但其推论在计算上非常昂贵。另一方面，我们在回答时使用更有效的策略：首先，如果我们可以自信地使用已经获得的知识来回答这个问题，那么我们甚至都不使用外部知识，而在我们确实需要外部知识的情况下，我们就不会一次阅读整个知识，而是只能阅读足够的知识来找到答案。在此程序的促进下，我们提出了一个研究问题：“开放域质量检查者可以像人类那样有效地利用外部知识而不牺牲预测表现？” 在这个问题的驱动下，我们探索了一种方法，该方法既利用了“封闭书”（利用模型参数中已经存在的知识）和“开放书籍”推断（利用外部知识）。此外，我们在多个“知识迭代”中动态读取外部知识，而不是使用大量固定段落进行开放式推理。通过对NQ和Triviaqa数据集进行的全面实验，我们证明了这种动态阅读方法提高了读者的“推理效率”和“预测准确性”。与FID阅读器相比，这种方法仅利用其读取器推理成本的18.32％来匹配其准确性，并且通过在NQ Open上实现高达55.10％的准确性来超越它。
Recent state-of-the-art open-domain QA models are typically based on a two stage retriever-reader approach in which the retriever first finds the relevant knowledge/passages and the reader then leverages that to predict the answer. Prior work has shown that the performance of the reader usually tends to improve with the increase in the number of these passages. Thus, state-of-the-art models use a large number of passages (e.g. 100) for inference. While the reader in this approach achieves high prediction performance, its inference is computationally very expensive. We humans, on the other hand, use a more efficient strategy while answering: firstly, if we can confidently answer the question using our already acquired knowledge then we do not even use the external knowledge, and in the case when we do require external knowledge, we don't read the entire knowledge at once, instead, we only read that much knowledge that is sufficient to find the answer. Motivated by this procedure, we ask a research question "Can the open-domain QA reader utilize external knowledge efficiently like humans without sacrificing the prediction performance?" Driven by this question, we explore an approach that utilizes both 'closed-book' (leveraging knowledge already present in the model parameters) and 'open-book' inference (leveraging external knowledge). Furthermore, instead of using a large fixed number of passages for open-book inference, we dynamically read the external knowledge in multiple 'knowledge iterations'. Through comprehensive experiments on NQ and TriviaQA datasets, we demonstrate that this dynamic reading approach improves both the 'inference efficiency' and the 'prediction accuracy' of the reader. Comparing with the FiD reader, this approach matches its accuracy by utilizing just 18.32% of its reader inference cost and also outperforms it by achieving up to 55.10% accuracy on NQ Open.
