Supervised Restricted Boltzmann Machines

Tu Dinh Nguyen, Dinh Phung, Viet Huynh, Trung Le
Proceedings of the 33rd Conference on Uncertainty in Artificial Intelligence, PMLR R15:780-789, 2017.

Abstract

We propose in this paper the supervised re- stricted Boltzmann machine (sRBM), a unified framework which combines the versatility of RBM to simultaneously learn the data represen- tation and to perform supervised learning (i.e., a nonlinear classifier or a nonlinear regressor). Un- like the current state-of-the-art classification for- mulation proposed for RBM in (Larochelle et al., 2012), our model is a hybrid probabilistic graph- ical model consisting of a distinguished genera- tive component for data representation and a dis- criminative component for prediction. While the work of (Larochelle et al., 2012) typically incurs no extra difficulty in inference compared with a standard RBM, our discriminative component, modeled as a directed graphical model, renders MCMC-based inference (e.g., Gibbs sampler) very slow and unpractical for use. To this end, we further develop scalable variational inference for the proposed sRBM for both classification and regression cases. Extensive experiments on real- world datasets show that our sRBM achieves bet- ter predictive performance than baseline meth- ods. At the same time, our proposed framework yields learned representations which are more discriminative, hence interpretable, than those of its counterparts. Besides, our method is prob- abilistic and capable of generating meaningful data conditioning on specific classes – a topic which is of current great interest in deep learn- ing aiming at data generation.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR15-nguyen17b, title = {Supervised Restricted {B}oltzmann Machines}, author = {Nguyen, Tu Dinh and Phung, Dinh and Huynh, Viet and Le, Trung}, booktitle = {Proceedings of the 33rd Conference on Uncertainty in Artificial Intelligence}, pages = {780--789}, year = {2017}, editor = {Elidan, Gal and Kersting, Kristian}, volume = {R15}, series = {Proceedings of Machine Learning Research}, month = {11--15 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r15/main/assets/nguyen17b/nguyen17b.pdf}, url = {https://proceedings.mlr.press/r15/nguyen17b.html}, abstract = {We propose in this paper the supervised re- stricted Boltzmann machine (sRBM), a unified framework which combines the versatility of RBM to simultaneously learn the data represen- tation and to perform supervised learning (i.e., a nonlinear classifier or a nonlinear regressor). Un- like the current state-of-the-art classification for- mulation proposed for RBM in (Larochelle et al., 2012), our model is a hybrid probabilistic graph- ical model consisting of a distinguished genera- tive component for data representation and a dis- criminative component for prediction. While the work of (Larochelle et al., 2012) typically incurs no extra difficulty in inference compared with a standard RBM, our discriminative component, modeled as a directed graphical model, renders MCMC-based inference (e.g., Gibbs sampler) very slow and unpractical for use. To this end, we further develop scalable variational inference for the proposed sRBM for both classification and regression cases. Extensive experiments on real- world datasets show that our sRBM achieves bet- ter predictive performance than baseline meth- ods. At the same time, our proposed framework yields learned representations which are more discriminative, hence interpretable, than those of its counterparts. Besides, our method is prob- abilistic and capable of generating meaningful data conditioning on specific classes – a topic which is of current great interest in deep learn- ing aiming at data generation.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T Supervised Restricted Boltzmann Machines %A Tu Dinh Nguyen %A Dinh Phung %A Viet Huynh %A Trung Le %B Proceedings of the 33rd Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2017 %E Gal Elidan %E Kristian Kersting %F pmlr-vR15-nguyen17b %I PMLR %P 780--789 %U https://proceedings.mlr.press/r15/nguyen17b.html %V R15 %X We propose in this paper the supervised re- stricted Boltzmann machine (sRBM), a unified framework which combines the versatility of RBM to simultaneously learn the data represen- tation and to perform supervised learning (i.e., a nonlinear classifier or a nonlinear regressor). Un- like the current state-of-the-art classification for- mulation proposed for RBM in (Larochelle et al., 2012), our model is a hybrid probabilistic graph- ical model consisting of a distinguished genera- tive component for data representation and a dis- criminative component for prediction. While the work of (Larochelle et al., 2012) typically incurs no extra difficulty in inference compared with a standard RBM, our discriminative component, modeled as a directed graphical model, renders MCMC-based inference (e.g., Gibbs sampler) very slow and unpractical for use. To this end, we further develop scalable variational inference for the proposed sRBM for both classification and regression cases. Extensive experiments on real- world datasets show that our sRBM achieves bet- ter predictive performance than baseline meth- ods. At the same time, our proposed framework yields learned representations which are more discriminative, hence interpretable, than those of its counterparts. Besides, our method is prob- abilistic and capable of generating meaningful data conditioning on specific classes – a topic which is of current great interest in deep learn- ing aiming at data generation. %Z Reissued by PMLR on 04 October 2026.
APA
Nguyen, T.D., Phung, D., Huynh, V. & Le, T.. (2017). Supervised Restricted Boltzmann Machines. Proceedings of the 33rd Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R15:780-789 Available from https://proceedings.mlr.press/r15/nguyen17b.html. Reissued by PMLR on 04 October 2026.

Related Material