Parallel Markov Chain Monte Carlo for Nonparametric Mixture Models

Sinead Williamson, Avinava Dubey, Eric Xing
Proceedings of the 30th International Conference on Machine Learning, PMLR 28(1):98-106, 2013.

Abstract

Nonparametric mixture models based on the Dirichlet process are an elegant alternative to finite models when the number of underlying components is unknown, but inference in such models can be slow. Existing attempts to parallelize inference in such models have relied on introducing approximations, which can lead to inaccuracies in the posterior estimate. In this paper, we describe auxiliary variable representations for the Dirichlet process and the hierarchical Dirichlet process that allow us to perform MCMC using the correct equilibrium distribution, in a distributed manner. We show that our approach allows scalable inference without the deterioration in estimate quality that accompanies existing methods.

Cite this Paper

BibTeX
@InProceedings{pmlr-v28-williamson13, title = {Parallel {M}arkov Chain {M}onte {C}arlo for Nonparametric Mixture Models}, author = {Williamson, Sinead and Dubey, Avinava and Xing, Eric}, booktitle = {Proceedings of the 30th International Conference on Machine Learning}, pages = {98--106}, year = {2013}, editor = {Dasgupta, Sanjoy and McAllester, David}, volume = {28}, number = {1}, series = {Proceedings of Machine Learning Research}, address = {Atlanta, Georgia, USA}, month = {17--19 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v28/williamson13.pdf}, url = {https://proceedings.mlr.press/v28/williamson13.html}, abstract = {Nonparametric mixture models based on the Dirichlet process are an elegant alternative to finite models when the number of underlying components is unknown, but inference in such models can be slow. Existing attempts to parallelize inference in such models have relied on introducing approximations, which can lead to inaccuracies in the posterior estimate. In this paper, we describe auxiliary variable representations for the Dirichlet process and the hierarchical Dirichlet process that allow us to perform MCMC using the correct equilibrium distribution, in a distributed manner. We show that our approach allows scalable inference without the deterioration in estimate quality that accompanies existing methods.} }
Endnote
%0 Conference Paper %T Parallel Markov Chain Monte Carlo for Nonparametric Mixture Models %A Sinead Williamson %A Avinava Dubey %A Eric Xing %B Proceedings of the 30th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2013 %E Sanjoy Dasgupta %E David McAllester %F pmlr-v28-williamson13 %I PMLR %P 98--106 %U https://proceedings.mlr.press/v28/williamson13.html %V 28 %N 1 %X Nonparametric mixture models based on the Dirichlet process are an elegant alternative to finite models when the number of underlying components is unknown, but inference in such models can be slow. Existing attempts to parallelize inference in such models have relied on introducing approximations, which can lead to inaccuracies in the posterior estimate. In this paper, we describe auxiliary variable representations for the Dirichlet process and the hierarchical Dirichlet process that allow us to perform MCMC using the correct equilibrium distribution, in a distributed manner. We show that our approach allows scalable inference without the deterioration in estimate quality that accompanies existing methods.
RIS
TY - CPAPER TI - Parallel Markov Chain Monte Carlo for Nonparametric Mixture Models AU - Sinead Williamson AU - Avinava Dubey AU - Eric Xing BT - Proceedings of the 30th International Conference on Machine Learning DA - 2013/02/13 ED - Sanjoy Dasgupta ED - David McAllester ID - pmlr-v28-williamson13 PB - PMLR DP - Proceedings of Machine Learning Research VL - 28 IS - 1 SP - 98 EP - 106 L1 - http://proceedings.mlr.press/v28/williamson13.pdf UR - https://proceedings.mlr.press/v28/williamson13.html AB - Nonparametric mixture models based on the Dirichlet process are an elegant alternative to finite models when the number of underlying components is unknown, but inference in such models can be slow. Existing attempts to parallelize inference in such models have relied on introducing approximations, which can lead to inaccuracies in the posterior estimate. In this paper, we describe auxiliary variable representations for the Dirichlet process and the hierarchical Dirichlet process that allow us to perform MCMC using the correct equilibrium distribution, in a distributed manner. We show that our approach allows scalable inference without the deterioration in estimate quality that accompanies existing methods. ER -
APA
Williamson, S., Dubey, A. & Xing, E.. (2013). Parallel Markov Chain Monte Carlo for Nonparametric Mixture Models. Proceedings of the 30th International Conference on Machine Learning, in Proceedings of Machine Learning Research 28(1):98-106 Available from https://proceedings.mlr.press/v28/williamson13.html.

Related Material