论文标题
Unicase-在语言模型中重新思考套管
UniCase -- Rethinking Casing in Language Models
论文作者
论文摘要
在本文中，我们介绍了一种新的方法来处理语言建模（LM）中的病例敏感性问题。我们将简单的体系结构修改为Roberta语言模型，并伴随着一种新的令牌化策略，我们将其命名为Unified Case LM（Unicase）。我们在胶水基准上测试了解决方案，这导致性能提高了0.42点。此外，我们证明，当我们必须处理所有令牌的文本数据（+5.88点）时，UNICASE模型的效果要好得多。
In this paper, we introduce a new approach to dealing with the problem of case-sensitiveness in Language Modelling (LM). We propose simple architecture modification to the RoBERTa language model, accompanied by a new tokenization strategy, which we named Unified Case LM (UniCase). We tested our solution on the GLUE benchmark, which led to increased performance by 0.42 points. Moreover, we prove that the UniCase model works much better when we have to deal with text data, where all tokens are uppercased (+5.88 point).
