论文标题
最终任务的改造结构感知的变压器语言模型
Retrofitting Structure-aware Transformer Language Model for End Tasks
论文作者
论文摘要
我们考虑通过提议利用句法距离来编码短语选区和依赖关系连接到语言模型来促进最终任务的改造，以促进最终任务。在多任务学习方案下，通过主要的语义任务培训来实现一种中层结构学习策略，以进行结构整合。实验结果表明，经过翻新的结构感知变压器语言模型可提高周期性，同时诱导准确的句法短语。通过执行结构意识的微调，我们的模型可以为语义依赖性和句法依赖性任务带来重大改进。
We consider retrofitting structure-aware Transformer-based language model for facilitating end tasks by proposing to exploit syntactic distance to encode both the phrasal constituency and dependency connection into the language model. A middle-layer structural learning strategy is leveraged for structure integration, accomplished with main semantic task training under multi-task learning scheme. Experimental results show that the retrofitted structure-aware Transformer language model achieves improved perplexity, meanwhile inducing accurate syntactic phrases. By performing structure-aware fine-tuning, our model achieves significant improvements for both semantic- and syntactic-dependent tasks.
