论文标题
G5：用于图形转移和启示录的通用图形 -
G5: A Universal GRAPH-BERT for Graph-to-Graph Transfer and Apocalypse Learning
论文作者
论文摘要
最近的Graph-Bert模型仅根据注意机制引入了一种学习图表的新方法。 Graph-bert提供了一个机会，可以在同一图数据集中传输跨不同任务的预训练模型和学习的图表表示。在本文中，我们将进一步研究跨不同图形数据集的通用图表学习的通用图形 - 伯特的图形传输，我们提出的模型也称为G5，为简单起见。学习G5中存在许多挑战，以适应每个图数据源的独特输入和输出配置以及信息分布差异。 G5引入了可插入的模型体系结构：（a）每个数据源将通过唯一的输入表示组件进行预处理； （b）每个输出应用程序任务也将具有特定的功能组件； （c）所有这些多样化的输入和输出组件将分别通过输入大小统一层和输出表示融合层与通用图形核心组件结合在一起。 G5模型消除了交叉绘图表示学习和转移的最后障碍。对于具有非常稀疏的训练数据的图源，在其他图上预先训练的G5模型仍可以通过必要的微调来表示表示。此外，G5的体系结构还使我们能够在没有任何培训数据的情况下学习有监督的功能分类器。在本文中，这样的问题也被称为启示录学习任务。本文介绍了两种不同的标签推理策略，即跨源分类一致性最大化（CCCM）和跨源动态路由（CDR），以解决该问题。
The recent GRAPH-BERT model introduces a new approach to learning graph representations merely based on the attention mechanism. GRAPH-BERT provides an opportunity for transferring pre-trained models and learned graph representations across different tasks within the same graph dataset. In this paper, we will further investigate the graph-to-graph transfer of a universal GRAPH-BERT for graph representation learning across different graph datasets, and our proposed model is also referred to as the G5 for simplicity. Many challenges exist in learning G5 to adapt the distinct input and output configurations for each graph data source, as well as the information distributions differences. G5 introduces a pluggable model architecture: (a) each data source will be pre-processed with a unique input representation learning component; (b) each output application task will also have a specific functional component; and (c) all such diverse input and output components will all be conjuncted with a universal GRAPH-BERT core component via an input size unification layer and an output representation fusion layer, respectively. The G5 model removes the last obstacle for cross-graph representation learning and transfer. For the graph sources with very sparse training data, the G5 model pre-trained on other graphs can still be utilized for representation learning with necessary fine-tuning. What's more, the architecture of G5 also allows us to learn a supervised functional classifier for data sources without any training data at all. Such a problem is also named as the Apocalypse Learning task in this paper. Two different label reasoning strategies, i.e., Cross-Source Classification Consistency Maximization (CCCM) and Cross-Source Dynamic Routing (CDR), are introduced in this paper to address the problem.
