论文标题
快速矩阵方形根，并应用于高斯流程和贝叶斯优化
Fast Matrix Square Roots with Applications to Gaussian Processes and Bayesian Optimization
论文作者
论文摘要
矩阵方形的根及其对逆，例如，在机器学习中，例如，当从高维高斯$ \ mathcal {n}（\ Mathbf 0，\ mathbf k）中采样时，或者使vector $ \ m $ \ mathbf b $相对于协方差矩阵$ \ mathbf k $。虽然现有方法通常需要$ o（n^3）$计算，但我们引入了一种用于计算$ \ MATHBF K^{1/2} \ MATHBF B $，$ \ MATHBF K^{ - 1/2} { - 1/2} \ MATHBF B $及其通过Matrix-Vector Multiletionation（Mat）的高效二次时间算法。我们的方法将Krylov子空间方法与合理的近似结合在一起，通常达到$ 4 $小数的准确度，不到$ 100 $ MVMS。此外，向后通行证几乎不需要额外的计算。我们证明了我们的方法在矩阵上的适用性高达$ 50，\！000 \ times 50，\！000 $ - 远远超出了传统方法 - 近似错误。将这种提高的可伸缩性应用于各种高斯工艺，贝叶斯优化和吉布斯进行采样，从而导致具有更高精度的更强大的模型。
Matrix square roots and their inverses arise frequently in machine learning, e.g., when sampling from high-dimensional Gaussians $\mathcal{N}(\mathbf 0, \mathbf K)$ or whitening a vector $\mathbf b$ against covariance matrix $\mathbf K$. While existing methods typically require $O(N^3)$ computation, we introduce a highly-efficient quadratic-time algorithm for computing $\mathbf K^{1/2} \mathbf b$, $\mathbf K^{-1/2} \mathbf b$, and their derivatives through matrix-vector multiplication (MVMs). Our method combines Krylov subspace methods with a rational approximation and typically achieves $4$ decimal places of accuracy with fewer than $100$ MVMs. Moreover, the backward pass requires little additional computation. We demonstrate our method's applicability on matrices as large as $50,\!000 \times 50,\!000$ - well beyond traditional methods - with little approximation error. Applying this increased scalability to variational Gaussian processes, Bayesian optimization, and Gibbs sampling results in more powerful models with higher accuracy.
