论文标题
通过铰链损失隐含最大化边缘
Implicitly Maximizing Margins with the Hinge Loss
论文作者
论文摘要
针对分类任务的神经网络提出了新的损失函数，该任务通过将梯度分配给其关键点扩展了铰链损失。我们将证明，对于具有固定步长的线性分离数据的线性分类器，此修改后的铰链损耗的边距将$ \ ell_2 $ max-margin收敛到$ \ mathcal {o}（1/t）$。与$ \ MATHCAL {O}（1/\ LOG T）$指数损失（如Logistic损失）相比，此速率很快。此外，经验结果表明，这种提高的收敛速度将延续到Relu网络。
A new loss function is proposed for neural networks on classification tasks which extends the hinge loss by assigning gradients to its critical points. We will show that for a linear classifier on linearly separable data with fixed step size, the margin of this modified hinge loss converges to the $\ell_2$ max-margin at the rate of $\mathcal{O}( 1/t )$. This rate is fast when compared with the $\mathcal{O}(1/\log t)$ rate of exponential losses such as the logistic loss. Furthermore, empirical results suggest that this increased convergence speed carries over to ReLU networks.
