Importance Weighted Consensus Monte Carlo for Distributed Bayesian Inference

Qiang Liu Dartmouth College
Proceedings of the 32nd Conference on Uncertainty in Artificial Intelligence, PMLR R14:632-641, 2016.

Abstract

The recent explosion in big data has created a significant challenge for efficient and scalable Bayesian inference. In this paper, we consider a divide-and-conquer setting in which the data is partitioned into different subsets with communication constraints, and a proper combination strategy is used to aggregate the Monte Carlo samples drawn from the local posteriors based on the dataset subsets. We propose a new importance weighted consensus Monte Carlo method for efficient Bayesian inference in this setting. Our method outperforms the previous one-shot combination strategies in terms of accuracy, and is more computation- and communication-efficient than the previous iterative combination methods that require iterative re-sampling and communication steps. We provide two practical versions of our approach, and illustrate their properties both theoretically and empirically.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR14-college16b, title = {Importance Weighted Consensus {M}onte {C}arlo for Distributed {B}ayesian Inference}, author = {College, Qiang Liu Dartmouth}, booktitle = {Proceedings of the 32nd Conference on Uncertainty in Artificial Intelligence}, pages = {632--641}, year = {2016}, editor = {Ihler, Alexander and Janzing, Dominik}, volume = {R14}, series = {Proceedings of Machine Learning Research}, month = {25--29 Jun}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r14/main/assets/college16b/college16b.pdf}, url = {https://proceedings.mlr.press/r14/college16b.html}, abstract = {The recent explosion in big data has created a significant challenge for efficient and scalable Bayesian inference. In this paper, we consider a divide-and-conquer setting in which the data is partitioned into different subsets with communication constraints, and a proper combination strategy is used to aggregate the Monte Carlo samples drawn from the local posteriors based on the dataset subsets. We propose a new importance weighted consensus Monte Carlo method for efficient Bayesian inference in this setting. Our method outperforms the previous one-shot combination strategies in terms of accuracy, and is more computation- and communication-efficient than the previous iterative combination methods that require iterative re-sampling and communication steps. We provide two practical versions of our approach, and illustrate their properties both theoretically and empirically.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T Importance Weighted Consensus Monte Carlo for Distributed Bayesian Inference %A Qiang Liu Dartmouth College %B Proceedings of the 32nd Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2016 %E Alexander Ihler %E Dominik Janzing %F pmlr-vR14-college16b %I PMLR %P 632--641 %U https://proceedings.mlr.press/r14/college16b.html %V R14 %X The recent explosion in big data has created a significant challenge for efficient and scalable Bayesian inference. In this paper, we consider a divide-and-conquer setting in which the data is partitioned into different subsets with communication constraints, and a proper combination strategy is used to aggregate the Monte Carlo samples drawn from the local posteriors based on the dataset subsets. We propose a new importance weighted consensus Monte Carlo method for efficient Bayesian inference in this setting. Our method outperforms the previous one-shot combination strategies in terms of accuracy, and is more computation- and communication-efficient than the previous iterative combination methods that require iterative re-sampling and communication steps. We provide two practical versions of our approach, and illustrate their properties both theoretically and empirically. %Z Reissued by PMLR on 04 October 2026.
APA
College, Q.L.D.. (2016). Importance Weighted Consensus Monte Carlo for Distributed Bayesian Inference. Proceedings of the 32nd Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R14:632-641 Available from https://proceedings.mlr.press/r14/college16b.html. Reissued by PMLR on 04 October 2026.

Related Material