Population Empirical Bayes

Alp Kucukelbir Columbia University, David Blei Columbia University
Proceedings of the 31st Conference on Uncertainty in Artificial Intelligence, PMLR R13:119-128, 2015.

Abstract

Bayesian predictive inference employs a model to analyze a dataset and make predictions about new observations. When a model does not match the data, predictive accuracy suffers. We develop population empirical Bayes, a hierarchical framework that explicitly models the empirical population distribution as part of Bayesian analysis. We introduce a latent dataset as a hierarchical variable and set the empirical population as its prior. This leads to a new predictive density that mitigates model mismatch. We efficiently apply this method to complex models by proposing a stochastic variational inference algorithm, called bumping variational inference. We demonstrate improved predictive accuracy over classical Bayesian inference in three models: a linear regression model of health data, a Bayesian mixture model of natural images, and a latent Dirichlet allocation topic model of a text corpus.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR13-university15b, title = {Population Empirical {B}ayes}, author = {University, Alp Kucukelbir Columbia and University, David Blei Columbia}, booktitle = {Proceedings of the 31st Conference on Uncertainty in Artificial Intelligence}, pages = {119--128}, year = {2015}, editor = {Meila, Marina and Heskes, Tom}, volume = {R13}, series = {Proceedings of Machine Learning Research}, month = {12--16 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r13/main/assets/university15b/university15b.pdf}, url = {https://proceedings.mlr.press/r13/university15b.html}, abstract = {Bayesian predictive inference employs a model to analyze a dataset and make predictions about new observations. When a model does not match the data, predictive accuracy suffers. We develop population empirical Bayes, a hierarchical framework that explicitly models the empirical population distribution as part of Bayesian analysis. We introduce a latent dataset as a hierarchical variable and set the empirical population as its prior. This leads to a new predictive density that mitigates model mismatch. We efficiently apply this method to complex models by proposing a stochastic variational inference algorithm, called bumping variational inference. We demonstrate improved predictive accuracy over classical Bayesian inference in three models: a linear regression model of health data, a Bayesian mixture model of natural images, and a latent Dirichlet allocation topic model of a text corpus.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T Population Empirical Bayes %A Alp Kucukelbir Columbia University %A David Blei Columbia University %B Proceedings of the 31st Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2015 %E Marina Meila %E Tom Heskes %F pmlr-vR13-university15b %I PMLR %P 119--128 %U https://proceedings.mlr.press/r13/university15b.html %V R13 %X Bayesian predictive inference employs a model to analyze a dataset and make predictions about new observations. When a model does not match the data, predictive accuracy suffers. We develop population empirical Bayes, a hierarchical framework that explicitly models the empirical population distribution as part of Bayesian analysis. We introduce a latent dataset as a hierarchical variable and set the empirical population as its prior. This leads to a new predictive density that mitigates model mismatch. We efficiently apply this method to complex models by proposing a stochastic variational inference algorithm, called bumping variational inference. We demonstrate improved predictive accuracy over classical Bayesian inference in three models: a linear regression model of health data, a Bayesian mixture model of natural images, and a latent Dirichlet allocation topic model of a text corpus. %Z Reissued by PMLR on 04 October 2026.
APA
University, A.K.C. & University, D.B.C.. (2015). Population Empirical Bayes. Proceedings of the 31st Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R13:119-128 Available from https://proceedings.mlr.press/r13/university15b.html. Reissued by PMLR on 04 October 2026.

Related Material