论文标题
具有退火重要性抽样的RBM的分区功能的有效评估
Efficient Evaluation of the Partition Function of RBMs with Annealed Importance Sampling
论文作者
论文摘要
基于受限玻尔兹曼机器（RBMS）的概率模型意味着评估归一化玻尔兹曼因子，这又需要对分区函数z的评估进行评估。但是，随着系统尺寸的增加，Z的确切评估成为了一个昂贵的任务。当人们考虑了RBM的最常见的学习算法时，这甚至会恶化，其中数据的经验分布的对数可能的梯度的确切评估包括每次迭代时Z的计算。退火重要性采样（AIS）方法提供了一种工具来随机估计系统的分区功能。到目前为止，在机器学习环境中，AIS算法的标准使用是使用大量蒙特卡洛步骤完成的。在这项工作中，我们表明，如果将适当的启动概率分布作为AIS算法的初始化，则可能不需要。我们在小型和大型问题中分析了AIS的性能，并表明在这两种情况下，可以在几乎没有计算成本的情况下获得Z的良好估计。
Probabilistic models based on Restricted Boltzmann Machines (RBMs) imply the evaluation of normalized Boltzmann factors, which in turn require from the evaluation of the partition function Z. The exact evaluation of Z, though, becomes a forbiddingly expensive task as the system size increases. This even worsens when one considers most usual learning algorithms for RBMs, where the exact evaluation of the gradient of the log-likelihood of the empirical distribution of the data includes the computation of Z at each iteration. The Annealed Importance Sampling (AIS) method provides a tool to stochastically estimate the partition function of the system. So far, the standard use of the AIS algorithm in the Machine Learning context has been done using a large number of Monte Carlo steps. In this work we show that this may not be required if a proper starting probability distribution is employed as the initialization of the AIS algorithm. We analyze the performance of AIS in both small- and large-sized problems, and show that in both cases a good estimation of Z can be obtained with little computational cost.
