论文标题
查看被忽视的：对自然语言推论的单词重叠偏见的分析
Looking at the Overlooked: An Analysis on the Word-Overlap Bias in Natural Language Inference
论文作者
论文摘要
已经表明，NLI模型通常相对于前提和假设之间的单词误链有偏见。他们以此功能为预测元素标签的主要提示。在本文中，我们专注于NLI模型中重叠偏差的一个被忽视的方面：反向单词跨层偏置。我们的实验结果表明，当前的NLI模型对低重叠的实例的非执行标签高度偏见，据报道，现有的偏见方法在现有的挑战数据集中取得了成功，通常无法在解决这一类别的偏见方面无效。我们研究了重叠偏见的出现的原因以及少数族裔在缓解中的作用。对于前者而言，我们发现单词重叠的偏见不是源于预训练，而对于后者，我们观察到，与公认的假设相反，消除少数族裔示例并不影响与重叠偏置相对于重叠偏见的概括方法的普遍性。
It has been shown that NLI models are usually biased with respect to the word-overlap between premise and hypothesis; they take this feature as a primary cue for predicting the entailment label. In this paper, we focus on an overlooked aspect of the overlap bias in NLI models: the reverse word-overlap bias. Our experimental results demonstrate that current NLI models are highly biased towards the non-entailment label on instances with low overlap, and the existing debiasing methods, which are reportedly successful on existing challenge datasets, are generally ineffective in addressing this category of bias. We investigate the reasons for the emergence of the overlap bias and the role of minority examples in its mitigation. For the former, we find that the word-overlap bias does not stem from pre-training, and for the latter, we observe that in contrast to the accepted assumption, eliminating minority examples does not affect the generalizability of debiasing methods with respect to the overlap bias.
