论文标题
在低系统资源环境中，转移学习模型与传统神经网络的性能
Performance of Transfer Learning Model vs. Traditional Neural Network in Low System Resource Environment
论文作者
论文摘要
最近，使用预训练的模型基于转移学习方法构建神经网络，越来越流行。这些预训练的模型具有使用较少的计算资源来培训较少培训数据的模型的好处。诸如BERT，XLNET和GPT之类的最新模型的兴起提高了准确性并作为转移倾斜的基础模型的好处。但是，这些模型仍然太复杂了，并且消耗了许多计算资源来训练以低GPU内存的转移学习。我们将比较较轻的转移学习模型和故意构建神经网络之间的性能和成本，以应用文本分类和NER模型。
Recently, the use of pre-trained model to build neural network based on transfer learning methodology is increasingly popular. These pre-trained models present the benefit of using less computing resources to train model with smaller amount of training data. The rise of state-of-the-art models such as BERT, XLNet and GPT boost accuracy and benefit as a base model for transfer leanring. However, these models are still too complex and consume many computing resource to train for transfer learning with low GPU memory. We will compare the performance and cost between lighter transfer learning model and purposely built neural network for NLP application of text classification and NER model.
