论文标题
将单语模型转移到低资源语言：Tigrinya的情况
Transferring Monolingual Model to Low-Resource Language: The Case of Tigrinya
论文作者
论文摘要
近年来，变压器模型在自然语言处理（NLP）任务方面取得了巨大成功。当前最新的NLP结果都是通过使用单语变压器模型来实现的，该模型使用单语言未标记的文本语料库进行了预训练。然后，该模型对特定的下游任务进行了微调。但是，对于大多数语言，预训练新变压器模型的成本很高。在这项工作中，我们提出了一种具有成本效益的转移学习方法，以采用强大的源语言模型，该模型从大型单语语料库训练到低资源语言。因此，使用XLNET语言模型，我们与Mbert展示了竞争性能，并在跨语言情感（CLS）数据集以及针对低资源性语言Tigrinya的新的情感分析数据集上展示了预先训练的目标语言模型。只有给定的Tigrinya情感分析数据集的10K示例，英语XLNet的F1得分分别超过了Bert和Mbert的78.88％，分别以10％和7％的速度。更有趣的是，与Mbert相比，CLS数据集中的微调（英语）XLNET模型具有令人鼓舞的结果，并且在一种日语的一个数据集中表现出色。
In recent years, transformer models have achieved great success in natural language processing (NLP) tasks. Most of the current state-of-the-art NLP results are achieved by using monolingual transformer models, where the model is pre-trained using a single language unlabelled text corpus. Then, the model is fine-tuned to the specific downstream task. However, the cost of pre-training a new transformer model is high for most languages. In this work, we propose a cost-effective transfer learning method to adopt a strong source language model, trained from a large monolingual corpus to a low-resource language. Thus, using XLNet language model, we demonstrate competitive performance with mBERT and a pre-trained target language model on the cross-lingual sentiment (CLS) dataset and on a new sentiment analysis dataset for low-resourced language Tigrinya. With only 10k examples of the given Tigrinya sentiment analysis dataset, English XLNet has achieved 78.88% F1-Score outperforming BERT and mBERT by 10% and 7%, respectively. More interestingly, fine-tuning (English) XLNet model on the CLS dataset has promising results compared to mBERT and even outperformed mBERT for one dataset of the Japanese language.
