论文标题
将依赖树整合到句子表示的自我注意力中
Integrating Dependency Tree Into Self-attention for Sentence Representation
论文作者
论文摘要
句子表示学习的解析树编码器的最新进展是显着的。但是，这些作品主要是递归地编码树结构，这不利于并行化。另一方面，这些作品很少考虑依赖树中弧的标签。为了解决这两个问题，我们提出了依赖性转化器，该转换器采用了一种与自我注意机制协同合作的关系 - 注意机制。该机制旨在编码依赖性句子树中节点之间的依赖关系和空间位置关系。通过基于分数的方法，我们成功地注入语法信息而不会影响变形金刚的并行性。我们的模型的表现优于句子表示的四个任务的最新方法，并且在计算效率方面具有明显的优势。
Recent progress on parse tree encoder for sentence representation learning is notable. However, these works mainly encode tree structures recursively, which is not conducive to parallelization. On the other hand, these works rarely take into account the labels of arcs in dependency trees. To address both issues, we propose Dependency-Transformer, which applies a relation-attention mechanism that works in concert with the self-attention mechanism. This mechanism aims to encode the dependency and the spatial positional relations between nodes in the dependency tree of sentences. By a score-based method, we successfully inject the syntax information without affecting Transformer's parallelizability. Our model outperforms or is comparable to the state-of-the-art methods on four tasks for sentence representation and has obvious advantages in computational efficiency.
