论文标题
语言模型可以具体吗？如何？
Can Language Models Be Specific? How?
论文作者
论文摘要
“他是一个人”，“巴黎位于地球上”。两种陈述都是正确的，但由于缺乏特异性而毫无意义。在本文中，我们建议衡量预先训练的语言模型（PLM）的特定语言。为了实现这一目标，我们引入了一种新颖的方法，通过用提示来形成蒙面的令牌预测任务来构建特异性测试的基准。例如，给定的“多伦多位于[面具]中。从我们的评估中，我们表明现有的PLM对更具体的答案只有一点偏好。我们确定影响特异性的潜在因素，并设计两种基于及时的方法以提高特异性。结果表明，在没有其他培训的情况下，建议的方法可以改善模型的特异性。我们希望这项工作能够使语言模型的特殊性概念意识到，并鼓励研究界进一步探索这个重要但经过研究的问题。
"He is a person", "Paris is located on the earth". Both statements are correct but meaningless - due to lack of specificity. In this paper, we propose to measure how specific the language of pre-trained language models (PLMs) is. To achieve this, we introduce a novel approach to build a benchmark for specificity testing by forming masked token prediction tasks with prompts. For instance, given "Toronto is located in [MASK].", we want to test whether a more specific answer will be better filled in by PLMs, e.g., Ontario instead of Canada. From our evaluations, we show that existing PLMs have only a slight preference for more specific answers. We identify underlying factors affecting the specificity and design two prompt-based methods to improve the specificity. Results show that the specificity of the models can be improved by the proposed methods without additional training. We hope this work can bring to awareness the notion of specificity of language models and encourage the research community to further explore this important but understudied problem.
