Bottleneck Conditional Density Estimation

Rui Shu, Hung H. Bui, Mohammad Ghavamzadeh
Proceedings of the 34th International Conference on Machine Learning, PMLR 70:3164-3172, 2017.

Abstract

We introduce a new framework for training deep generative models for high-dimensional conditional density estimation. The Bottleneck Conditional Density Estimator (BCDE) is a variant of the conditional variational autoencoder (CVAE) that employs layer(s) of stochastic variables as the bottleneck between the input x and target y, where both are high-dimensional. Crucially, we propose a new hybrid training method that blends the conditional generative model with a joint generative model. Hybrid blending is the key to effective training of the BCDE, which avoids overfitting and provides a novel mechanism for leveraging unlabeled data. We show that our hybrid training procedure enables models to achieve competitive results in the MNIST quadrant prediction task in the fully-supervised setting, and sets new benchmarks in the semi-supervised regime for MNIST, SVHN, and CelebA.

Cite this Paper

BibTeX
@InProceedings{pmlr-v70-shu17a, title = {Bottleneck Conditional Density Estimation}, author = {Rui Shu and Hung H. Bui and Mohammad Ghavamzadeh}, booktitle = {Proceedings of the 34th International Conference on Machine Learning}, pages = {3164--3172}, year = {2017}, editor = {Precup, Doina and Teh, Yee Whye}, volume = {70}, series = {Proceedings of Machine Learning Research}, month = {06--11 Aug}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v70/shu17a/shu17a.pdf}, url = {https://proceedings.mlr.press/v70/shu17a.html}, abstract = {We introduce a new framework for training deep generative models for high-dimensional conditional density estimation. The Bottleneck Conditional Density Estimator (BCDE) is a variant of the conditional variational autoencoder (CVAE) that employs layer(s) of stochastic variables as the bottleneck between the input x and target y, where both are high-dimensional. Crucially, we propose a new hybrid training method that blends the conditional generative model with a joint generative model. Hybrid blending is the key to effective training of the BCDE, which avoids overfitting and provides a novel mechanism for leveraging unlabeled data. We show that our hybrid training procedure enables models to achieve competitive results in the MNIST quadrant prediction task in the fully-supervised setting, and sets new benchmarks in the semi-supervised regime for MNIST, SVHN, and CelebA.} }
Endnote
%0 Conference Paper %T Bottleneck Conditional Density Estimation %A Rui Shu %A Hung H. Bui %A Mohammad Ghavamzadeh %B Proceedings of the 34th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2017 %E Doina Precup %E Yee Whye Teh %F pmlr-v70-shu17a %I PMLR %P 3164--3172 %U https://proceedings.mlr.press/v70/shu17a.html %V 70 %X We introduce a new framework for training deep generative models for high-dimensional conditional density estimation. The Bottleneck Conditional Density Estimator (BCDE) is a variant of the conditional variational autoencoder (CVAE) that employs layer(s) of stochastic variables as the bottleneck between the input x and target y, where both are high-dimensional. Crucially, we propose a new hybrid training method that blends the conditional generative model with a joint generative model. Hybrid blending is the key to effective training of the BCDE, which avoids overfitting and provides a novel mechanism for leveraging unlabeled data. We show that our hybrid training procedure enables models to achieve competitive results in the MNIST quadrant prediction task in the fully-supervised setting, and sets new benchmarks in the semi-supervised regime for MNIST, SVHN, and CelebA.
APA
Shu, R., Bui, H.H. & Ghavamzadeh, M.. (2017). Bottleneck Conditional Density Estimation. Proceedings of the 34th International Conference on Machine Learning, in Proceedings of Machine Learning Research 70:3164-3172 Available from https://proceedings.mlr.press/v70/shu17a.html.

Related Material