论文标题
通过克服灾难性遗忘的现代方法计算出的神经网络权重的重要性的相关性
Correlation of the importances of neural network weights calculated by modern methods of overcoming catastrophic forgetting
论文作者
论文摘要
在2017年EWC方法的发明之后，已经提出了几种方法来计算神经网络权重在EWC方法中使用的重要性。尽管计算权重的重要性有显着差异，但它们都被证明是有效的。因此，出现了一个合理的问题，即通过不同方法计算的权重的重要性如何相似。为了回答这个问题，我们计算了所有这些方法计算的权重的重要性的逐层相关性。结果，事实证明，几种方法的重要性彼此相互密切相关，我们能够为这种相关性提供解释。同时，对于其他方法，相关性可能从网络的某些层上的强度到其他层的负面。这提出了一个合理的问题：为什么，尽管计算方法非常不同，但所有这些重要性都允许EWC方法能够完全克服灾难性忘记神经网络？
Following the invention in 2017 of the EWC method, several methods have been proposed to calculate the importance of neural network weights for use in the EWC method. Despite the significant difference in calculating the importance of weights, they all proved to be effective. Accordingly, a reasonable question arises as to how similar the importances of the weights calculated by different methods. To answer this question, we calculated layer-by-layer correlations of the importance of weights calculated by all those methods. As a result, it turned out that the importances of several of the methods correlated with each other quite strongly and we were able to present an explanation for such a correlation. At the same time, for other methods, the correlation can vary from strong on some layers of the network to negative on other layers. Which raises a reasonable question: why, despite the very different calculation methods, all those importances allow EWC method to overcome the catastrophic forgetting of neural networks perfectly?
