论文标题
通过解决普通微分方程来培训生成对抗网络
Training Generative Adversarial Networks by Solving Ordinary Differential Equations
论文作者
论文摘要
生成对抗网络（GAN）训练的不稳定性经常归因于梯度下降。因此，最近的方法旨在调整模型和培训程序以稳定离散更新。相反，我们研究了GAN训练引起的连续时间动力学。理论和玩具实验都表明这些动力学实际上是令人惊讶的稳定。从这个角度来看，我们假设训练中的不稳定性是由于离散连续动态的整合误差引起的。我们通过实验验证，当与控制集成误差的正规机构结合使用时，众所周知的ODE求解器（例如Runge-Kutta）可以稳定训练。我们的方法代表了与以前的方法的根本性不同，该方法通常使用自适应优化和稳定技术来限制功能空间（例如光谱归一化）。对CIFAR-10和ImageNet的评估表明，我们的方法表现出了几个强大的基准，证明其功效。
The instability of Generative Adversarial Network (GAN) training has frequently been attributed to gradient descent. Consequently, recent methods have aimed to tailor the models and training procedures to stabilise the discrete updates. In contrast, we study the continuous-time dynamics induced by GAN training. Both theory and toy experiments suggest that these dynamics are in fact surprisingly stable. From this perspective, we hypothesise that instabilities in training GANs arise from the integration error in discretising the continuous dynamics. We experimentally verify that well-known ODE solvers (such as Runge-Kutta) can stabilise training - when combined with a regulariser that controls the integration error. Our approach represents a radical departure from previous methods which typically use adaptive optimisation and stabilisation techniques that constrain the functional space (e.g. Spectral Normalisation). Evaluation on CIFAR-10 and ImageNet shows that our method outperforms several strong baselines, demonstrating its efficacy.
