论文标题
Textbrewer：一种用于自然语言处理的开源知识蒸馏工具包
TextBrewer: An Open-Source Knowledge Distillation Toolkit for Natural Language Processing
论文作者
论文摘要
在本文中，我们介绍了Textbrewer，这是一种用于自然语言处理的开源知识蒸馏工具包。它可以与不同的神经网络模型一起使用，并支持各种监督的学习任务，例如文本分类，阅读理解，序列标签。 TextBrewer提供了一个简单且均匀的工作流程，可快速设置具有高度灵活配置的蒸馏实验。它提供了一组预定义的蒸馏方法，可以使用自定义代码扩展。作为一个案例研究，我们使用Textbrewer将BERT蒸馏出几个典型的NLP任务。通过简单的配置，我们实现了与具有相似参数数量的公共蒸馏BERT模型相当甚至更高的结果。我们的工具包可通过以下方式获得：http：//textbrewer.hfl-rc.com
In this paper, we introduce TextBrewer, an open-source knowledge distillation toolkit designed for natural language processing. It works with different neural network models and supports various kinds of supervised learning tasks, such as text classification, reading comprehension, sequence labeling. TextBrewer provides a simple and uniform workflow that enables quick setting up of distillation experiments with highly flexible configurations. It offers a set of predefined distillation methods and can be extended with custom code. As a case study, we use TextBrewer to distill BERT on several typical NLP tasks. With simple configurations, we achieve results that are comparable with or even higher than the public distilled BERT models with similar numbers of parameters. Our toolkit is available through: http://textbrewer.hfl-rc.com
