论文标题
旨在鲁克西克数据集偏见稳健地对NLI模型进行稳固
Towards Robustifying NLI Models Against Lexical Dataset Biases
论文作者
论文摘要
虽然深度学习模型在自然语言推理的任务上取得了快速的进步，但最近的研究还表明，这些模型通过利用多种数据集偏见而没有深入了解语言语义来实现高精度。本文将使用矛盾字偏置和单词重叠偏置作为我们的两个偏见示例，探讨了数据级和模型级别的偏差方法，以鲁克西克数据集偏见为鲁克斯式模型进行鲁棒性。首先，我们通过数据增强和增强数据集DEBIAS，但表明无法通过此方法完全删除模型偏差。接下来，我们还比较了两种直接对模型进行偏见的方法，而又不知道数据集偏差是什么。第一种方法旨在消除嵌入级别的标签偏差。第二种方法采用字袋子模型来捕获可能利用偏见并防止原始模型通过强迫这两个子模型之间的正交性学习这些偏见特征的功能。我们对从原始MNLI数据集和NLI应力测试提取的新的平衡数据集进行了评估，并表明正交性方法在模型的同时保持竞争性总体准确性时更好。我们的代码和数据可在以下网址找到：https：//github.com/owenzx/lexicaldebias-acl2020
While deep learning models are making fast progress on the task of Natural Language Inference, recent studies have also shown that these models achieve high accuracy by exploiting several dataset biases, and without deep understanding of the language semantics. Using contradiction-word bias and word-overlapping bias as our two bias examples, this paper explores both data-level and model-level debiasing methods to robustify models against lexical dataset biases. First, we debias the dataset through data augmentation and enhancement, but show that the model bias cannot be fully removed via this method. Next, we also compare two ways of directly debiasing the model without knowing what the dataset biases are in advance. The first approach aims to remove the label bias at the embedding level. The second approach employs a bag-of-words sub-model to capture the features that are likely to exploit the bias and prevents the original model from learning these biased features by forcing orthogonality between these two sub-models. We performed evaluations on new balanced datasets extracted from the original MNLI dataset as well as the NLI stress tests, and show that the orthogonality approach is better at debiasing the model while maintaining competitive overall accuracy. Our code and data are available at: https://github.com/owenzx/LexicalDebias-ACL2020
