论文标题
Equibormar：3D原子图
Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphs
论文作者
论文摘要
尽管在各个领域取得了广泛的成功，但即使考虑到3D与3D相关的电感偏差（如翻译不变性和旋转式率），也考虑到3D原子图域（例如分子）的域（例如分子）的跨数据集的表现良好。在本文中，我们证明了变压器可以很好地概括为3D原子图和当前的Equibormer，这是一个图形神经网络利用变压器体系结构的强度，并结合了基于不可征服（IRREPS）的SE（3）/E（3）/E（3） - 等级特征。首先，我们仅通过用量量的产品替换了变形金刚中的原始操作，提出了一种简单有效的体系结构。使用Equivariant操作可以在不复杂的图形结构复杂化的情况下，在IRREPS功能的通道中编码Equivariant信息。随着对变压器的最小修改，该体系结构已经取得了强大的经验结果。其次，我们提出了一种名为Equivariast Graph注意的新型注意机制，该机制通过用多层式感知器的注意力替换DOT产品的注意力，在变压器中的典型注意力中提高了典型的注意，并包括非线性消息传递。通过这两项创新，Equibrouner在QM9，MD17和OC20数据集上的先前模型中取得了竞争成果。
Despite their widespread success in various domains, Transformer networks have yet to perform well across datasets in the domain of 3D atomistic graphs such as molecules even when 3D-related inductive biases like translational invariance and rotational equivariance are considered. In this paper, we demonstrate that Transformers can generalize well to 3D atomistic graphs and present Equiformer, a graph neural network leveraging the strength of Transformer architectures and incorporating SE(3)/E(3)-equivariant features based on irreducible representations (irreps). First, we propose a simple and effective architecture by only replacing original operations in Transformers with their equivariant counterparts and including tensor products. Using equivariant operations enables encoding equivariant information in channels of irreps features without complicating graph structures. With minimal modifications to Transformers, this architecture has already achieved strong empirical results. Second, we propose a novel attention mechanism called equivariant graph attention, which improves upon typical attention in Transformers through replacing dot product attention with multi-layer perceptron attention and including non-linear message passing. With these two innovations, Equiformer achieves competitive results to previous models on QM9, MD17 and OC20 datasets.
