Austerity in MCMC Land: Cutting the Metropolis-Hastings Budget

Anoop Korattikara, Yutian Chen, Max Welling
Proceedings of the 31st International Conference on Machine Learning, PMLR 32(1):181-189, 2014.

Abstract

Can we make Bayesian posterior MCMC sampling more efficient when faced with very large datasets? We argue that computing the likelihood for N datapoints in the Metropolis-Hastings (MH) test to reach a single binary decision is computationally inefficient. We introduce an approximate MH rule based on a sequential hypothesis test that allows us to accept or reject samples with high confidence using only a fraction of the data required for the exact MH rule. While this method introduces an asymptotic bias, we show that this bias can be controlled and is more than offset by a decrease in variance due to our ability to draw more samples per unit of time.

Cite this Paper

BibTeX
@InProceedings{pmlr-v32-korattikara14, title = {Austerity in MCMC Land: Cutting the Metropolis-Hastings Budget}, author = {Korattikara, Anoop and Chen, Yutian and Welling, Max}, booktitle = {Proceedings of the 31st International Conference on Machine Learning}, pages = {181--189}, year = {2014}, editor = {Xing, Eric P. and Jebara, Tony}, volume = {32}, number = {1}, series = {Proceedings of Machine Learning Research}, address = {Bejing, China}, month = {22--24 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v32/korattikara14.pdf}, url = {https://proceedings.mlr.press/v32/korattikara14.html}, abstract = {Can we make Bayesian posterior MCMC sampling more efficient when faced with very large datasets? We argue that computing the likelihood for N datapoints in the Metropolis-Hastings (MH) test to reach a single binary decision is computationally inefficient. We introduce an approximate MH rule based on a sequential hypothesis test that allows us to accept or reject samples with high confidence using only a fraction of the data required for the exact MH rule. While this method introduces an asymptotic bias, we show that this bias can be controlled and is more than offset by a decrease in variance due to our ability to draw more samples per unit of time.} }
Endnote
%0 Conference Paper %T Austerity in MCMC Land: Cutting the Metropolis-Hastings Budget %A Anoop Korattikara %A Yutian Chen %A Max Welling %B Proceedings of the 31st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2014 %E Eric P. Xing %E Tony Jebara %F pmlr-v32-korattikara14 %I PMLR %P 181--189 %U https://proceedings.mlr.press/v32/korattikara14.html %V 32 %N 1 %X Can we make Bayesian posterior MCMC sampling more efficient when faced with very large datasets? We argue that computing the likelihood for N datapoints in the Metropolis-Hastings (MH) test to reach a single binary decision is computationally inefficient. We introduce an approximate MH rule based on a sequential hypothesis test that allows us to accept or reject samples with high confidence using only a fraction of the data required for the exact MH rule. While this method introduces an asymptotic bias, we show that this bias can be controlled and is more than offset by a decrease in variance due to our ability to draw more samples per unit of time.
RIS
TY - CPAPER TI - Austerity in MCMC Land: Cutting the Metropolis-Hastings Budget AU - Anoop Korattikara AU - Yutian Chen AU - Max Welling BT - Proceedings of the 31st International Conference on Machine Learning DA - 2014/01/27 ED - Eric P. Xing ED - Tony Jebara ID - pmlr-v32-korattikara14 PB - PMLR DP - Proceedings of Machine Learning Research VL - 32 IS - 1 SP - 181 EP - 189 L1 - http://proceedings.mlr.press/v32/korattikara14.pdf UR - https://proceedings.mlr.press/v32/korattikara14.html AB - Can we make Bayesian posterior MCMC sampling more efficient when faced with very large datasets? We argue that computing the likelihood for N datapoints in the Metropolis-Hastings (MH) test to reach a single binary decision is computationally inefficient. We introduce an approximate MH rule based on a sequential hypothesis test that allows us to accept or reject samples with high confidence using only a fraction of the data required for the exact MH rule. While this method introduces an asymptotic bias, we show that this bias can be controlled and is more than offset by a decrease in variance due to our ability to draw more samples per unit of time. ER -
APA
Korattikara, A., Chen, Y. & Welling, M.. (2014). Austerity in MCMC Land: Cutting the Metropolis-Hastings Budget. Proceedings of the 31st International Conference on Machine Learning, in Proceedings of Machine Learning Research 32(1):181-189 Available from https://proceedings.mlr.press/v32/korattikara14.html.

Related Material