论文标题
强烈凸功能的最快一阶优化方法的高分辨率建模
High-Resolution Modeling of the Fastest First-Order Optimization Method for Strongly Convex Functions
论文作者
论文摘要
通过限制普通微分方程（ODE）的角度研究基于梯度的优化算法的事实，在这里，我们得出了加速三重动量（TM）算法的ODE表示。对于不受限制的优化问题，其凸成本强劲，TM算法的收敛速度比Nesterov的加速梯度（NAG）方法更快，但具有相同的计算复杂性。我们显示，类似于NAG方法的准确捕获TM方法的特征，我们需要使用高分辨率建模来获得TM算法的ode表示。我们使用Lyapunov分析来研究TM算法提出的高分辨率ODE表示的稳定性和收敛行为。我们通过该分析表明，该ode模型偏离TM算法的参数具有鲁棒性。我们将TM方法的ODE表示速率与NAG方法的ode表示速率进行了比较，以确认其更快的收敛性。我们的研究还导致了NAG方法的ODE模型的最差收敛速度的更严格的结合。最后，我们讨论了积分二次约束（IQC）方法的使用来建立对TM算法收敛速率的估计。一个数字示例证明了我们的结果。
Motivated by the fact that the gradient-based optimization algorithms can be studied from the perspective of limiting ordinary differential equations (ODEs), here we derive an ODE representation of the accelerated triple momentum (TM) algorithm. For unconstrained optimization problems with strongly convex cost, the TM algorithm has a proven faster convergence rate than the Nesterov's accelerated gradient (NAG) method but with the same computational complexity. We show that similar to the NAG method to capture accurately the characteristics of the TM method, we need to use a high-resolution modeling to obtain the ODE representation of the TM algorithm. We use a Lyapunov analysis to investigate the stability and convergence behavior of the proposed high-resolution ODE representation of the TM algorithm. We show through this analysis that this ODE model has robustness to deviation from the parameters of the TM algorithm. We compare the rate of the ODE representation of the TM method with that of the NAG method to confirm its faster convergence. Our study also leads to a tighter bound on the worst rate of convergence for the ODE model of the NAG method. Lastly, we discuss the use of the integral quadratic constraint (IQC) method to establish an estimate on the rate of convergence of the TM algorithm. A numerical example demonstrates our results.
