论文标题
HADONLI：探索自然语言推断的仅假设偏见的人造模式
HypoNLI: Exploring the Artificial Patterns of Hypothesis-only Bias in Natural Language Inference
论文作者
论文摘要
许多最近的研究表明，对于在自然语言推断数据集中训练的模型（NLI），可以通过仅查看该假设，同时完全忽略前提来做出正确的预测。在这项工作中，我们设法从仅假设的偏见方面得出了对抗性例子，并探索了减轻这种偏见的合格方法。具体而言，我们从训练集中的假设（人造模式）中提取各种短语，并表明它们是特定标签的强烈指标。然后，我们从标签与这些指示相反或一致的原始测试集中找出“硬”和“简单”实例。我们还建立了包括审计模型（Bert，Roberta，XLNet）和竞争性非预言模型（Infersent，DAM，ESIM）的基准。除了基准和基线外，我们还研究了两种利用人工模型建模以减轻这种仅假设偏见的偏见方法：下采样和对抗性训练。我们认为，这些方法可以被视为NLI偏见任务中的竞争基线。
Many recent studies have shown that for models trained on datasets for natural language inference (NLI), it is possible to make correct predictions by merely looking at the hypothesis while completely ignoring the premise. In this work, we manage to derive adversarial examples in terms of the hypothesis-only bias and explore eligible ways to mitigate such bias. Specifically, we extract various phrases from the hypotheses (artificial patterns) in the training sets, and show that they have been strong indicators to the specific labels. We then figure out `hard' and `easy' instances from the original test sets whose labels are opposite to or consistent with those indications. We also set up baselines including both pretrained models (BERT, RoBERTa, XLNet) and competitive non-pretrained models (InferSent, DAM, ESIM). Apart from the benchmark and baselines, we also investigate two debiasing approaches which exploit the artificial pattern modeling to mitigate such hypothesis-only bias: down-sampling and adversarial training. We believe those methods can be treated as competitive baselines in NLI debiasing tasks.
