论文标题
分解中的变分深度搜索
Variational Depth Search in ResNets
论文作者
论文摘要
单发神经架构搜索允许联合学习权重和网络体系结构，从而降低了计算成本。我们将搜索空间限制在残留网络的深度上，并制定一个可分析的可拖动变异目标，该目标允许在一击中获得无偏见的后端。我们提出了一个启发式方法，以根据此分布修剪我们的网络。我们将我们提出的方法与MNIST，时尚和SVHN数据集的网络深度进行手动搜索进行了比较。我们发现，修剪的网络不会蒙受预测性能的损失，从而获得了与未经修复的网络竞争的准确性。边缘化的深度使我们能够在单个正向通行证中获得比常规网络更好地校准的测试时间不确定性估计。
One-shot neural architecture search allows joint learning of weights and network architecture, reducing computational cost. We limit our search space to the depth of residual networks and formulate an analytically tractable variational objective that allows for obtaining an unbiased approximate posterior over depths in one-shot. We propose a heuristic to prune our networks based on this distribution. We compare our proposed method against manual search over network depths on the MNIST, Fashion-MNIST, SVHN datasets. We find that pruned networks do not incur a loss in predictive performance, obtaining accuracies competitive with unpruned networks. Marginalising over depth allows us to obtain better-calibrated test-time uncertainty estimates than regular networks, in a single forward pass.
