Doubly Stochastic Variational Bayes for non-Conjugate Inference

Michalis Titsias, Miguel Lázaro-Gredilla
Proceedings of the 31st International Conference on Machine Learning, PMLR 32(2):1971-1979, 2014.

Abstract

We propose a simple and effective variational inference algorithm based on stochastic optimisation that can be widely applied for Bayesian non-conjugate inference in continuous parameter spaces. This algorithm is based on stochastic approximation and allows for efficient use of gradient information from the model joint density. We demonstrate these properties using illustrative examples as well as in challenging and diverse Bayesian inference problems such as variable selection in logistic regression and fully Bayesian inference over kernel hyperparameters in Gaussian process regression.

Cite this Paper

BibTeX
@InProceedings{pmlr-v32-titsias14, title = {Doubly Stochastic Variational Bayes for non-Conjugate Inference}, author = {Titsias, Michalis and Lázaro-Gredilla, Miguel}, booktitle = {Proceedings of the 31st International Conference on Machine Learning}, pages = {1971--1979}, year = {2014}, editor = {Xing, Eric P. and Jebara, Tony}, volume = {32}, number = {2}, series = {Proceedings of Machine Learning Research}, address = {Bejing, China}, month = {22--24 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v32/titsias14.pdf}, url = {https://proceedings.mlr.press/v32/titsias14.html}, abstract = {We propose a simple and effective variational inference algorithm based on stochastic optimisation that can be widely applied for Bayesian non-conjugate inference in continuous parameter spaces. This algorithm is based on stochastic approximation and allows for efficient use of gradient information from the model joint density. We demonstrate these properties using illustrative examples as well as in challenging and diverse Bayesian inference problems such as variable selection in logistic regression and fully Bayesian inference over kernel hyperparameters in Gaussian process regression.} }
Endnote
%0 Conference Paper %T Doubly Stochastic Variational Bayes for non-Conjugate Inference %A Michalis Titsias %A Miguel Lázaro-Gredilla %B Proceedings of the 31st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2014 %E Eric P. Xing %E Tony Jebara %F pmlr-v32-titsias14 %I PMLR %P 1971--1979 %U https://proceedings.mlr.press/v32/titsias14.html %V 32 %N 2 %X We propose a simple and effective variational inference algorithm based on stochastic optimisation that can be widely applied for Bayesian non-conjugate inference in continuous parameter spaces. This algorithm is based on stochastic approximation and allows for efficient use of gradient information from the model joint density. We demonstrate these properties using illustrative examples as well as in challenging and diverse Bayesian inference problems such as variable selection in logistic regression and fully Bayesian inference over kernel hyperparameters in Gaussian process regression.
RIS
TY - CPAPER TI - Doubly Stochastic Variational Bayes for non-Conjugate Inference AU - Michalis Titsias AU - Miguel Lázaro-Gredilla BT - Proceedings of the 31st International Conference on Machine Learning DA - 2014/06/18 ED - Eric P. Xing ED - Tony Jebara ID - pmlr-v32-titsias14 PB - PMLR DP - Proceedings of Machine Learning Research VL - 32 IS - 2 SP - 1971 EP - 1979 L1 - http://proceedings.mlr.press/v32/titsias14.pdf UR - https://proceedings.mlr.press/v32/titsias14.html AB - We propose a simple and effective variational inference algorithm based on stochastic optimisation that can be widely applied for Bayesian non-conjugate inference in continuous parameter spaces. This algorithm is based on stochastic approximation and allows for efficient use of gradient information from the model joint density. We demonstrate these properties using illustrative examples as well as in challenging and diverse Bayesian inference problems such as variable selection in logistic regression and fully Bayesian inference over kernel hyperparameters in Gaussian process regression. ER -
APA
Titsias, M. & Lázaro-Gredilla, M.. (2014). Doubly Stochastic Variational Bayes for non-Conjugate Inference. Proceedings of the 31st International Conference on Machine Learning, in Proceedings of Machine Learning Research 32(2):1971-1979 Available from https://proceedings.mlr.press/v32/titsias14.html.

Related Material