Learning Deep Sigmoid Belief Networks with Data Augmentation

Zhe Gan, Ricardo Henao, David Carlson, Lawrence Carin
Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics, PMLR 38:268-276, 2015.

Abstract

Deep directed generative models are developed. The multi-layered model is designed by stacking sigmoid belief networks, with sparsity-encouraging priors placed on the model parameters. Learning and inference of layer-wise model parameters are implemented in a Bayesian setting. By exploring the idea of data augmentation and introducing auxiliary Polya-Gamma variables, simple and efficient Gibbs sampling and mean-field variational Bayes (VB) inference are implemented. To address large-scale datasets, an online version of VB is also developed. Experimental results are presented for three publicly available datasets: MNIST, Caltech 101 Silhouettes and OCR letters.

Cite this Paper

BibTeX
@InProceedings{pmlr-v38-gan15, title = {{Learning Deep Sigmoid Belief Networks with Data Augmentation}}, author = {Gan, Zhe and Henao, Ricardo and Carlson, David and Carin, Lawrence}, booktitle = {Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics}, pages = {268--276}, year = {2015}, editor = {Lebanon, Guy and Vishwanathan, S. V. N.}, volume = {38}, series = {Proceedings of Machine Learning Research}, address = {San Diego, California, USA}, month = {09--12 May}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v38/gan15.pdf}, url = {https://proceedings.mlr.press/v38/gan15.html}, abstract = {Deep directed generative models are developed. The multi-layered model is designed by stacking sigmoid belief networks, with sparsity-encouraging priors placed on the model parameters. Learning and inference of layer-wise model parameters are implemented in a Bayesian setting. By exploring the idea of data augmentation and introducing auxiliary Polya-Gamma variables, simple and efficient Gibbs sampling and mean-field variational Bayes (VB) inference are implemented. To address large-scale datasets, an online version of VB is also developed. Experimental results are presented for three publicly available datasets: MNIST, Caltech 101 Silhouettes and OCR letters.} }
Endnote
%0 Conference Paper %T Learning Deep Sigmoid Belief Networks with Data Augmentation %A Zhe Gan %A Ricardo Henao %A David Carlson %A Lawrence Carin %B Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2015 %E Guy Lebanon %E S. V. N. Vishwanathan %F pmlr-v38-gan15 %I PMLR %P 268--276 %U https://proceedings.mlr.press/v38/gan15.html %V 38 %X Deep directed generative models are developed. The multi-layered model is designed by stacking sigmoid belief networks, with sparsity-encouraging priors placed on the model parameters. Learning and inference of layer-wise model parameters are implemented in a Bayesian setting. By exploring the idea of data augmentation and introducing auxiliary Polya-Gamma variables, simple and efficient Gibbs sampling and mean-field variational Bayes (VB) inference are implemented. To address large-scale datasets, an online version of VB is also developed. Experimental results are presented for three publicly available datasets: MNIST, Caltech 101 Silhouettes and OCR letters.
RIS
TY - CPAPER TI - Learning Deep Sigmoid Belief Networks with Data Augmentation AU - Zhe Gan AU - Ricardo Henao AU - David Carlson AU - Lawrence Carin BT - Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics DA - 2015/02/21 ED - Guy Lebanon ED - S. V. N. Vishwanathan ID - pmlr-v38-gan15 PB - PMLR DP - Proceedings of Machine Learning Research VL - 38 SP - 268 EP - 276 L1 - http://proceedings.mlr.press/v38/gan15.pdf UR - https://proceedings.mlr.press/v38/gan15.html AB - Deep directed generative models are developed. The multi-layered model is designed by stacking sigmoid belief networks, with sparsity-encouraging priors placed on the model parameters. Learning and inference of layer-wise model parameters are implemented in a Bayesian setting. By exploring the idea of data augmentation and introducing auxiliary Polya-Gamma variables, simple and efficient Gibbs sampling and mean-field variational Bayes (VB) inference are implemented. To address large-scale datasets, an online version of VB is also developed. Experimental results are presented for three publicly available datasets: MNIST, Caltech 101 Silhouettes and OCR letters. ER -
APA
Gan, Z., Henao, R., Carlson, D. & Carin, L.. (2015). Learning Deep Sigmoid Belief Networks with Data Augmentation. Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 38:268-276 Available from https://proceedings.mlr.press/v38/gan15.html.

Related Material