论文标题
Maxpooling的理论表达
The Theoretical Expressiveness of Maxpooling
论文作者
论文摘要
自从深度神经网络成为最先进的图像分类器的状态以来，已经有一种倾向于减少最大池的趋势：在图像中占据附近最大像素的功能。由于Max Poling在早期的图像分类器中以突出的特征为特色，因此我们希望了解这一趋势，以及它是否合理。我们开发了一个理论框架，分析基于RELU的近似值以最大池，并证明无法使用Relu激活有效地复制最大池。我们分析了一类最佳近似值的误差，发现误差可以在内核大小上成倍小，但这需要成倍复杂的近似值。 我们的工作为理解趋势从最新架构中的最大池化提供了理论基础。我们得出的结论是，最大池和最佳近似之间存在差异的主要原因，池中最大值和其他值之间的巨大差异可以通过其他建筑决策来克服，或者在自然图像中不普遍。
Over the decade since deep neural networks became state of the art image classifiers there has been a tendency towards less use of max pooling: the function that takes the largest of nearby pixels in an image. Since max pooling featured prominently in earlier generations of image classifiers, we wish to understand this trend, and whether it is justified. We develop a theoretical framework analyzing ReLU based approximations to max pooling, and prove a sense in which max pooling cannot be efficiently replicated using ReLU activations. We analyze the error of a class of optimal approximations, and find that whilst the error can be made exponentially small in the kernel size, doing so requires an exponentially complex approximation. Our work gives a theoretical basis for understanding the trend away from max pooling in newer architectures. We conclude that the main cause of a difference between max pooling and an optimal approximation, a prevalent large difference between the max and other values within pools, can be overcome with other architectural decisions, or is not prevalent in natural images.
