论文标题
绿巨人：负责任的自然语言处理的能源效率基准平台
HULK: An Energy Efficiency Benchmark Platform for Responsible Natural Language Processing
论文作者
论文摘要
计算密集型预处理的模型一直在许多自然语言处理基准（例如胶水）的领导下。但是，模型训练和推断过程中的能源效率成为关键的瓶颈。我们介绍了绿巨人，这是负责任的自然语言处理的多任务能效基准平台。借助绿巨人，我们从时间和成本的角度比较了验证的模型的能源效率。为进一步分析提供了基线基准测试结果。不同任务之间的微调效率可能会有很大差异，而较少的参数编号并不一定意味着更好的效率。我们分析了这种现象，并证明了比较预验证模型的多任务效率的方法。我们的平台可从https://sites.engineering.ucsb.edu/~xiyou/hulk/获得。
Computation-intensive pretrained models have been taking the lead of many natural language processing benchmarks such as GLUE. However, energy efficiency in the process of model training and inference becomes a critical bottleneck. We introduce HULK, a multi-task energy efficiency benchmarking platform for responsible natural language processing. With HULK, we compare pretrained models' energy efficiency from the perspectives of time and cost. Baseline benchmarking results are provided for further analysis. The fine-tuning efficiency of different pretrained models can differ a lot among different tasks and fewer parameter number does not necessarily imply better efficiency. We analyzed such phenomenon and demonstrate the method of comparing the multi-task efficiency of pretrained models. Our platform is available at https://sites.engineering.ucsb.edu/~xiyou/hulk/.
