论文标题
ensmallen的灵活数值优化
Flexible numerical optimization with ensmallen
论文作者
论文摘要
该报告介绍了Ensmallen数值优化库，并深入了解了其工作原理的技术细节。该库为任意用户提供的功能的数学优化提供了快速，灵活的C ++框架。提供了大量的预构建优化器，包括许多随机梯度下降和准Newton优化器的变体。支持几种类型的目标功能，包括可区分，可分离，约束和分类目标功能。新优化器的实现只需要一种方法，而新的目标函数通常只需要一种或两个C ++方法。通过内部使用C ++模板元编程，Ensmallen为任意用户提供的回调提供了支持，并自动推断未填充的方法没有任何运行时开销。经验比较表明，Ensmallen的表现优于其他优化框架（例如Julia和Scipy），有时是通过很大的边缘。该库可在https://ensmallen.org上找到，并根据BSD许可证分发。
This report provides an introduction to the ensmallen numerical optimization library, as well as a deep dive into the technical details of how it works. The library provides a fast and flexible C++ framework for mathematical optimization of arbitrary user-supplied functions. A large set of pre-built optimizers is provided, including many variants of Stochastic Gradient Descent and Quasi-Newton optimizers. Several types of objective functions are supported, including differentiable, separable, constrained, and categorical objective functions. Implementation of a new optimizer requires only one method, while a new objective function requires typically only one or two C++ methods. Through internal use of C++ template metaprogramming, ensmallen provides support for arbitrary user-supplied callbacks and automatic inference of unsupplied methods without any runtime overhead. Empirical comparisons show that ensmallen outperforms other optimization frameworks (such as Julia and SciPy), sometimes by large margins. The library is available at https://ensmallen.org and is distributed under the permissive BSD license.
