论文标题
浅神经网络可以击败维度的诅咒吗？平均野外训练观点
Can Shallow Neural Networks Beat the Curse of Dimensionality? A mean field training perspective
论文作者
论文摘要
我们证明，在平均野外规模下，两层神经网络对经验或人口风险的梯度下降训练可能不会比$ t^{ - 4/（d-2）} $更快地降低人口风险。因此，梯度下降训练以合理的平滑拟合，但真正的高维数据可能会受到维度的诅咒。我们提供的数值证据表明，随着尺寸的增加，具有一般Lipschitz目标函数的梯度下降训练变得越来越慢，但是当目标函数位于两层relu网络的自然功能空间中时，在所有维度上的收敛速度大致相同。
We prove that the gradient descent training of a two-layer neural network on empirical or population risk may not decrease population risk at an order faster than $t^{-4/(d-2)}$ under mean field scaling. Thus gradient descent training for fitting reasonably smooth, but truly high-dimensional data may be subject to the curse of dimensionality. We present numerical evidence that gradient descent training with general Lipschitz target functions becomes slower and slower as the dimension increases, but converges at approximately the same rate in all dimensions when the target function lies in the natural function space for two-layer ReLU networks.
