论文标题
熵，自由能和受限玻尔兹曼机器的工作
Entropy, Free Energy, and Work of Restricted Boltzmann Machines
论文作者
论文摘要
受限的玻尔兹曼机器是一种生成概率的图形网络。在某种配置中查找网络的概率是由Boltzmann分布给出的。给定培训数据，其学习是通过优化网络能量函数的参数来完成的。在本文中，我们在统计物理学的背景下分析了受限玻尔兹曼机器的训练过程。作为说明，对于小尺寸的条形条纹模式，我们计算热力学数量，例如熵，自由能和内部能量，随训练时期的函数。随着训练的进行，我们通过熵的亚熵来证明可见层和隐藏层之间的相关性的增长。使用在配置空间中可见和隐藏向量的轨迹的蒙特卡洛模拟，我们还通过切换能量函数的参数来计算在受限的玻尔兹曼机器上完成的工作的分布。我们讨论了jarzynski平等，该平等连接工作的指数函数的路径平均值以及训练前后自由能的差异。
A restricted Boltzmann machine is a generative probabilistic graphic network. A probability of finding the network in a certain configuration is given by the Boltzmann distribution. Given training data, its learning is done by optimizing parameters of the energy function of the network. In this paper, we analyze the training process of the restricted Boltzmann machine in the context of statistical physics. As an illustration, for small size Bar-and-Stripe patterns, we calculate thermodynamic quantities such as entropy, free energy, and internal energy as a function of training epoch. We demonstrate the growth of the correlation between the visible and hidden layers via the subadditivity of entropies as the training proceeds. Using the Monte-Carlo simulation of trajectories of the visible and hidden vectors in configuration space, we also calculate the distribution of the work done on the restricted Boltzmann machine by switching the parameters of the energy function. We discuss the Jarzynski equality which connects the path average of the exponential function of the work and the difference in free energies before and after training.
