Inductive Principles for Restricted Boltzmann Machine Learning


Benjamin Marlin, Kevin Swersky, Bo Chen, Nando Freitas ;
Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, PMLR 9:509-516, 2010.


Recent research has seen the proposal of several new inductive principles designed specifically to avoid the problems associated with maximum likelihood learning in models with intractable partition functions. In this paper, we study learning methods for binary restricted Boltzmann machines (RBMs) based on ratio matching and generalized score matching. We compare these new RBM learning methods to a range of existing learning methods including stochastic maximum likelihood, contrastive divergence, and pseudo-likelihood. We perform an extensive empirical evaluation across multiple tasks and data sets.

Related Material