论文标题
研究BERT中的新型动词学习：选择性偏好类和基于交替的句法概括
Investigating Novel Verb Learning in BERT: Selectional Preference Classes and Alternation-Based Syntactic Generalization
论文作者
论文摘要
先前研究深度学习模型句法能力的研究并未针对语法概括的强度与模型在训练过程中暴露的证据数量之间的关系。我们通过部署一种新颖的单词学习范式来解决这个问题，以测试Bert为英语动词的两个方面的几个方面的学习能力：替代和选择性偏好的类别。对于前者，我们将Bert在言语偏置对的单个框架上微调，并询问该模型是否期望新颖的动词出现在其姐妹框架中。对于后者，我们在不完整的口头对象的选择网络上微调伯特，并询问它是否期望未进行但合理的动词/对象对。我们发现，伯特仅在微调中只有一个或两个新单词的实例，就可以进行强大的语法概括。对于言语交替测试，我们发现模型显示与传递性偏差一致的行为：预计几次的动词会采用直接对象，但是与直接对象看到的动词不会被无关紧要。
Previous studies investigating the syntactic abilities of deep learning models have not targeted the relationship between the strength of the grammatical generalization and the amount of evidence to which the model is exposed during training. We address this issue by deploying a novel word-learning paradigm to test BERT's few-shot learning capabilities for two aspects of English verbs: alternations and classes of selectional preferences. For the former, we fine-tune BERT on a single frame in a verbal-alternation pair and ask whether the model expects the novel verb to occur in its sister frame. For the latter, we fine-tune BERT on an incomplete selectional network of verbal objects and ask whether it expects unattested but plausible verb/object pairs. We find that BERT makes robust grammatical generalizations after just one or two instances of a novel word in fine-tuning. For the verbal alternation tests, we find that the model displays behavior that is consistent with a transitivity bias: verbs seen few times are expected to take direct objects, but verbs seen with direct objects are not expected to occur intransitively.
