论文标题
lipschitz连续损失功能的扰动迭代SGD
Perturbed Iterate SGD for Lipschitz Continuous Loss Functions
论文作者
论文摘要
本文介绍了随机梯度下降的扩展，以最大程度地减少Lipschitz的连续损失函数。我们的动机是用于非平滑非凸的随机优化问题，这些问题经常在机器学习等应用中遇到。使用Clarke $ε$ -subdifferential，我们证明了该方法的非反应收敛到预期的近似固定点。从此结果，几乎可以肯定地开发了一种具有高概率的非反应收敛的方法，以及一种具有渐近收敛到Clarke固定点的方法。我们的结果在假设的假设是，随机损耗函数是一种carathéodory函数，在决策变量中，Lipschitz几乎无处不在。据我们所知，这是在这些最小假设下的第一个非质合收敛分析。
This paper presents an extension of stochastic gradient descent for the minimization of Lipschitz continuous loss functions. Our motivation is for use in non-smooth non-convex stochastic optimization problems, which are frequently encountered in applications such as machine learning. Using the Clarke $ε$-subdifferential, we prove the non-asymptotic convergence to an approximate stationary point in expectation for the proposed method. From this result, a method with non-asymptotic convergence with high probability, as well as a method with asymptotic convergence to a Clarke stationary point almost surely are developed. Our results hold under the assumption that the stochastic loss function is a Carathéodory function which is almost everywhere Lipschitz continuous in the decision variables. To the best of our knowledge this is the first non-asymptotic convergence analysis under these minimal assumptions.
