Stochastic Structured Variational Inference

Matthew Hoffman, David Blei
Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics, PMLR 38:361-369, 2015.

Abstract

Stochastic variational inference makes it possible to approximate posterior distributions induced by large datasets quickly using stochastic optimization. The algorithm relies on the use of fully factorized variational distributions. However, this “mean-field” independence approximation limits the fidelity of the posterior approximation, and introduces local optima. We show how to relax the mean-field approximation to allow arbitrary dependencies between global parameters and local hidden variables, producing better parameter estimates by reducing bias, sensitivity to local optima, and sensitivity to hyperparameters.

Cite this Paper

BibTeX
@InProceedings{pmlr-v38-hoffman15, title = {{Stochastic Structured Variational Inference}}, author = {Hoffman, Matthew and Blei, David}, booktitle = {Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics}, pages = {361--369}, year = {2015}, editor = {Lebanon, Guy and Vishwanathan, S. V. N.}, volume = {38}, series = {Proceedings of Machine Learning Research}, address = {San Diego, California, USA}, month = {09--12 May}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v38/hoffman15.pdf}, url = {https://proceedings.mlr.press/v38/hoffman15.html}, abstract = {Stochastic variational inference makes it possible to approximate posterior distributions induced by large datasets quickly using stochastic optimization. The algorithm relies on the use of fully factorized variational distributions. However, this “mean-field” independence approximation limits the fidelity of the posterior approximation, and introduces local optima. We show how to relax the mean-field approximation to allow arbitrary dependencies between global parameters and local hidden variables, producing better parameter estimates by reducing bias, sensitivity to local optima, and sensitivity to hyperparameters.} }
Endnote
%0 Conference Paper %T Stochastic Structured Variational Inference %A Matthew Hoffman %A David Blei %B Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2015 %E Guy Lebanon %E S. V. N. Vishwanathan %F pmlr-v38-hoffman15 %I PMLR %P 361--369 %U https://proceedings.mlr.press/v38/hoffman15.html %V 38 %X Stochastic variational inference makes it possible to approximate posterior distributions induced by large datasets quickly using stochastic optimization. The algorithm relies on the use of fully factorized variational distributions. However, this “mean-field” independence approximation limits the fidelity of the posterior approximation, and introduces local optima. We show how to relax the mean-field approximation to allow arbitrary dependencies between global parameters and local hidden variables, producing better parameter estimates by reducing bias, sensitivity to local optima, and sensitivity to hyperparameters.
RIS
TY - CPAPER TI - Stochastic Structured Variational Inference AU - Matthew Hoffman AU - David Blei BT - Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics DA - 2015/02/21 ED - Guy Lebanon ED - S. V. N. Vishwanathan ID - pmlr-v38-hoffman15 PB - PMLR DP - Proceedings of Machine Learning Research VL - 38 SP - 361 EP - 369 L1 - http://proceedings.mlr.press/v38/hoffman15.pdf UR - https://proceedings.mlr.press/v38/hoffman15.html AB - Stochastic variational inference makes it possible to approximate posterior distributions induced by large datasets quickly using stochastic optimization. The algorithm relies on the use of fully factorized variational distributions. However, this “mean-field” independence approximation limits the fidelity of the posterior approximation, and introduces local optima. We show how to relax the mean-field approximation to allow arbitrary dependencies between global parameters and local hidden variables, producing better parameter estimates by reducing bias, sensitivity to local optima, and sensitivity to hyperparameters. ER -
APA
Hoffman, M. & Blei, D.. (2015). Stochastic Structured Variational Inference. Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 38:361-369 Available from https://proceedings.mlr.press/v38/hoffman15.html.

Related Material