论文标题
稀疏数据设置的域适应性：不使用bert我们会获得什么？
Domain Adaptation for Sparse-Data Settings: What Do We Gain by Not Using Bert?
论文作者
论文摘要
NLP的实际成功取决于培训数据的可用性。但是，在实际情况下，培训数据通常很少，尤其是因为许多应用程序域是受限制和特定的。在这项工作中，我们比较了处理此问题的不同方法，并在只有少量标记的培训数据可用于特定域时，为构建NLP应用程序提供指南。尽管使用预训练的语言模型转移学习胜过任务的其他方法，但替代方案的表现并不能力较差，同时需要减少计算工作，从而大大降低了货币和环境成本。我们检查了几种此类替代方案的性能权衡，包括可以更快地训练175,000次的模型，并且不需要单个GPU。
The practical success of much of NLP depends on the availability of training data. However, in real-world scenarios, training data is often scarce, not least because many application domains are restricted and specific. In this work, we compare different methods to handle this problem and provide guidelines for building NLP applications when there is only a small amount of labeled training data available for a specific domain. While transfer learning with pre-trained language models outperforms other methods across tasks, alternatives do not perform much worse while requiring much less computational effort, thus significantly reducing monetary and environmental cost. We examine the performance tradeoffs of several such alternatives, including models that can be trained up to 175K times faster and do not require a single GPU.
