论文标题
卷积和残留网络可证明包含彩票
Convolutional and Residual Networks Provably Contain Lottery Tickets
论文作者
论文摘要
彩票票证假设继续对对小规模深度神经网络的追求产生深远的实际影响，这些神经网络以竞争性能解决了现代深度学习任务。这些彩票是通过用与应用程序一样多样化的架构进行修剪的大型随机初始化神经网络来识别的。然而，理论上的见解证明了它们的存在，主要集中在具有Relu激活功能的深度完全连接的饲料前向网络上。我们证明，还可以配备几乎任意激活功能的卷积和残差层组成的现代体系结构也可能包含较高可能性的彩票。
The Lottery Ticket Hypothesis continues to have a profound practical impact on the quest for small scale deep neural networks that solve modern deep learning tasks at competitive performance. These lottery tickets are identified by pruning large randomly initialized neural networks with architectures that are as diverse as their applications. Yet, theoretical insights that attest their existence have been mostly focused on deep fully-connected feed forward networks with ReLU activation functions. We prove that also modern architectures consisting of convolutional and residual layers that can be equipped with almost arbitrary activation functions can contain lottery tickets with high probability.
