论文标题
零阶负曲率发现：逃脱鞍点没有梯度
Zeroth-Order Negative Curvature Finding: Escaping Saddle Points without Gradients
论文作者
论文摘要
我们考虑逃脱了仅访问函数评估的非凸问题的鞍点。尽管已经提出了各种作品，但其中大多数都需要第二阶或一阶信息，并且只有少数已经利用了零阶方法，尤其是使用零阶方法的负曲率发现技术，这些方法已被证明是逃脱鞍点的最有效方法。为了填补这一空白，在本文中，我们提出了两个零阶负曲率查找框架，这些框架可以替换黑西斯 - 矢量产品计算而不会增加迭代复杂性。我们将提出的框架应用于ZO-GD，ZO-SGD，ZO-SCSG，ZO-SPIDER，并证明这些ZO算法可以收敛到$（ε，δ）$ - 与以前的Zeroth-Zeroth-Zeroth-Zeroth-sort serfors一起查找本地最小值的工作相比，查询复杂性较小。
We consider escaping saddle points of nonconvex problems where only the function evaluations can be accessed. Although a variety of works have been proposed, the majority of them require either second or first-order information, and only a few of them have exploited zeroth-order methods, particularly the technique of negative curvature finding with zeroth-order methods which has been proven to be the most efficient method for escaping saddle points. To fill this gap, in this paper, we propose two zeroth-order negative curvature finding frameworks that can replace Hessian-vector product computations without increasing the iteration complexity. We apply the proposed frameworks to ZO-GD, ZO-SGD, ZO-SCSG, ZO-SPIDER and prove that these ZO algorithms can converge to $(ε,δ)$-approximate second-order stationary points with less query complexity compared with prior zeroth-order works for finding local minima.
