论文标题

加速梯度方法将Tikhonov正则化与由Hessian驱动的几何阻尼相结合

Accelerated gradient methods combining Tikhonov regularization with geometric damping driven by the Hessian

论文作者

Attouch, Hedy, Balhag, Aicha, Chbani, Zaki, Riahi, Hassan

论文摘要

在希尔伯特(Hilbert)的环境中,为了凸出优化,我们考虑将梯度动力学与tikhonov正则化与黑锡驱动阻尼相结合。由于时间趋向于无穷大,因此假定Tikhonov正则化参数趋于零,从而保留了平衡。 Tikhonov正则化项的存在诱导了渐近消失的强凸特性。为了利用在强凸状情况下附加到重球方法的指数收敛速率,我们考虑了惯性动态,其中粘性阻尼系数与Tikhonov正则化参数的平方根成正比,因此也将其收敛于零。此外,动态涉及几何阻尼,这是由要最小化的函数的Hessian驱动的,从而引起了振荡的显着衰减。在对参数的适当调整下,基于Lyapunov的分析,我们表明轨迹同时具有几种显着的属性:它们提供了值的快速收敛,梯度向零的快速收敛以及对最小规范最小化器的强收敛。这项研究扩展了作者先前的论文,其中检查了类似问题,但没有黑森西亚驱动阻尼的存在。

In a Hilbert setting, for convex differentiable optimization, we consider accelerated gradient dynamics combining Tikhonov regularization with Hessian-driven damping. The Tikhonov regularization parameter is assumed to tend to zero as time tends to infinity, which preserves equilibria. The presence of the Tikhonov regularization term induces a strong convexity property which vanishes asymptotically. To take advantage of the exponential convergence rates attached to the heavy ball method in the strongly convex case, we consider the inertial dynamic where the viscous damping coefficient is taken proportional to the square root of the Tikhonov regularization parameter, and therefore also converges towards zero. Moreover, the dynamic involves a geometric damping which is driven by the Hessian of the function to be minimized, which induces a significant attenuation of the oscillations. Under an appropriate tuning of the parameters, based on Lyapunov's analysis, we show that the trajectories have at the same time several remarkable properties: they provide fast convergence of values, fast convergence of gradients towards zero, and strong convergence to the minimum norm minimizer. This study extends a previous paper by the authors where similar issues were examined but without the presence of Hessian driven damping.

扫码加入交流群

加入微信交流群

微信交流群二维码

扫码加入学术交流群,获取更多资源