A Forest Mixture Bound for Block-Free Parallel Inference

Neal Lawton, Greg Ver Steeg, Aram Galstyan
Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence, PMLR R16:967-976, 2018.

Abstract

Coordinate ascent variational inference is an important algorithm for inference in probabilistic models, but it is slow because it updates only a single variable at a time. Block coordinate methods perform inference faster by updating blocks of variables in parallel. However, the speed and convergence of these algorithms depends on how the variables are partitioned into blocks. In this paper, we give a convergent parallel algorithm for inference in deep exponential families that doesn’t require the variables to be partitioned into blocks. We achieve this by lower bounding the ELBO by a new objective we call the forest mixture bound (FM bound) that separates the inference problem for variables within a hidden layer. We apply this to the simple case when all random variables are Gaussian and show empirically that the algorithm converges faster for models that are inherently more forest-like.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR16-lawton18a, title = {A Forest Mixture Bound for Block-Free Parallel Inference}, author = {Lawton, Neal and Steeg, Greg Ver and Galstyan, Aram}, booktitle = {Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence}, pages = {967--976}, year = {2018}, editor = {Globerson, Amir and Silva, Ricardo}, volume = {R16}, series = {Proceedings of Machine Learning Research}, month = {06--10 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r16/main/assets/lawton18a/lawton18a.pdf}, url = {https://proceedings.mlr.press/r16/lawton18a.html}, abstract = {Coordinate ascent variational inference is an important algorithm for inference in probabilistic models, but it is slow because it updates only a single variable at a time. Block coordinate methods perform inference faster by updating blocks of variables in parallel. However, the speed and convergence of these algorithms depends on how the variables are partitioned into blocks. In this paper, we give a convergent parallel algorithm for inference in deep exponential families that doesn’t require the variables to be partitioned into blocks. We achieve this by lower bounding the ELBO by a new objective we call the forest mixture bound (FM bound) that separates the inference problem for variables within a hidden layer. We apply this to the simple case when all random variables are Gaussian and show empirically that the algorithm converges faster for models that are inherently more forest-like.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T A Forest Mixture Bound for Block-Free Parallel Inference %A Neal Lawton %A Greg Ver Steeg %A Aram Galstyan %B Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2018 %E Amir Globerson %E Ricardo Silva %F pmlr-vR16-lawton18a %I PMLR %P 967--976 %U https://proceedings.mlr.press/r16/lawton18a.html %V R16 %X Coordinate ascent variational inference is an important algorithm for inference in probabilistic models, but it is slow because it updates only a single variable at a time. Block coordinate methods perform inference faster by updating blocks of variables in parallel. However, the speed and convergence of these algorithms depends on how the variables are partitioned into blocks. In this paper, we give a convergent parallel algorithm for inference in deep exponential families that doesn’t require the variables to be partitioned into blocks. We achieve this by lower bounding the ELBO by a new objective we call the forest mixture bound (FM bound) that separates the inference problem for variables within a hidden layer. We apply this to the simple case when all random variables are Gaussian and show empirically that the algorithm converges faster for models that are inherently more forest-like. %Z Reissued by PMLR on 04 October 2026.
APA
Lawton, N., Steeg, G.V. & Galstyan, A.. (2018). A Forest Mixture Bound for Block-Free Parallel Inference. Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R16:967-976 Available from https://proceedings.mlr.press/r16/lawton18a.html. Reissued by PMLR on 04 October 2026.

Related Material