A Spike and Slab Restricted Boltzmann Machine


Aaron Courville, James Bergstra, Yoshua Bengio ;
Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, PMLR 15:233-241, 2011.


We introduce the spike and slab Restricted Boltzmann Machine, characterized by having both a real-valued vector, the slab, and a binary variable, the spike, associated with each unit in the hidden layer. The model possesses some practical properties such as being amenable to Block Gibbs sampling as well as being capable of generating similar latent representations of the data to the recently introduced mean and covariance Restricted Boltzmann Machine. We illustrate how the spike and slab Restricted Boltzmann Machine achieves competitive performance on the CIFAR-10 object recognition task. [pdf]

Related Material