论文标题
神经网络的图形结构
Graph Structure of Neural Networks
论文作者
论文摘要
神经网络通常表示为神经元之间的连接图。但是，尽管它们广泛使用，但目前对神经网络的图形结构及其预测性能之间的关系几乎没有理解。在这里，我们系统地研究神经网络的图结构如何影响其预测性能。为此，我们开发了一种基于图形的神经网络的新型表示，称为关系图，其中神经网络计算的层对应于沿图结构的消息交换的回合。使用此表示，我们表明：（1）关系图的“最佳点”导致神经网络具有显着改善的预测性能； （2）神经网络的性能大致是其关系图的聚类系数和平均路径长度的平滑函数； （3）我们的发现在许多不同的任务和数据集中是一致的； （4）可以有效地确定甜点； （5）表现最佳的神经网络具有与真实生物神经网络的图形结构令人惊讶地相似。我们的工作为设计神经体系结构的设计和对神经网络的理解打开了新的方向。
Neural networks are often represented as graphs of connections between neurons. However, despite their wide use, there is currently little understanding of the relationship between the graph structure of the neural network and its predictive performance. Here we systematically investigate how does the graph structure of neural networks affect their predictive performance. To this end, we develop a novel graph-based representation of neural networks called relational graph, where layers of neural network computation correspond to rounds of message exchange along the graph structure. Using this representation we show that: (1) a "sweet spot" of relational graphs leads to neural networks with significantly improved predictive performance; (2) neural network's performance is approximately a smooth function of the clustering coefficient and average path length of its relational graph; (3) our findings are consistent across many different tasks and datasets; (4) the sweet spot can be identified efficiently; (5) top-performing neural networks have graph structure surprisingly similar to those of real biological neural networks. Our work opens new directions for the design of neural architectures and the understanding on neural networks in general.
