论文标题
具有自然梯度下降的图形神经网络的优化
Optimization of Graph Neural Networks with Natural Gradient Descent
论文作者
论文摘要
在这项工作中，我们建议采用信息几何工具来优化图形神经网络体系结构，例如图形卷积网络。更具体地说，我们通过在优化过程中采用自然梯度信息来为基于图的半监督学习开发优化算法。这使我们能够有效利用基础统计模型或参数空间的几何形状进行优化和推理。据我们所知，这是第一项利用自然梯度来优化图神经网络的工作，可以扩展到其他半监督问题。开发了有效的计算算法，并进行了广泛的数值研究，以证明我们的算法优于ADAM和SGD等现有算法。
In this work, we propose to employ information-geometric tools to optimize a graph neural network architecture such as the graph convolutional networks. More specifically, we develop optimization algorithms for the graph-based semi-supervised learning by employing the natural gradient information in the optimization process. This allows us to efficiently exploit the geometry of the underlying statistical model or parameter space for optimization and inference. To the best of our knowledge, this is the first work that has utilized the natural gradient for the optimization of graph neural networks that can be extended to other semi-supervised problems. Efficient computations algorithms are developed and extensive numerical studies are conducted to demonstrate the superior performance of our algorithms over existing algorithms such as ADAM and SGD.
