A Forest Mixture Bound for Block-Free Parallel Inference

Neal Lawton, Greg Ver Steeg, Aram Galstyan
Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence, PMLR R16:967-976, 2018.

Abstract

Coordinate ascent variational inference is an important algorithm for inference in proba- bilistic models, but it is slow because it updates only a single variable at a time. Block coordi- nate methods perform inference faster by up- dating blocks of variables in parallel. How- ever, the speed and convergence of these algo- rithms depends on how the variables are par- titioned into blocks. In this paper, we give a convergent parallel algorithm for inference in deep exponential families that doesn’t require the variables to be partitioned into blocks. We achieve this by lower bounding the ELBO by a new objective we call the forest mixture bound (FM bound) that separates the inference prob- lem for variables within a hidden layer. We apply this to the simple case when all random variables are Gaussian and show empirically that the algorithm converges faster for models that are inherently more forest-like.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR16-lawton18a, title = {A Forest Mixture Bound for Block-Free Parallel Inference}, author = {Lawton, Neal and Steeg, Greg Ver and Galstyan, Aram}, booktitle = {Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence}, pages = {967--976}, year = {2018}, editor = {Globerson, Amir and Silva, Ricardo}, volume = {R16}, series = {Proceedings of Machine Learning Research}, month = {06--10 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r16/main/assets/lawton18a/lawton18a.pdf}, url = {https://proceedings.mlr.press/r16/lawton18a.html}, abstract = {Coordinate ascent variational inference is an important algorithm for inference in proba- bilistic models, but it is slow because it updates only a single variable at a time. Block coordi- nate methods perform inference faster by up- dating blocks of variables in parallel. How- ever, the speed and convergence of these algo- rithms depends on how the variables are par- titioned into blocks. In this paper, we give a convergent parallel algorithm for inference in deep exponential families that doesn’t require the variables to be partitioned into blocks. We achieve this by lower bounding the ELBO by a new objective we call the forest mixture bound (FM bound) that separates the inference prob- lem for variables within a hidden layer. We apply this to the simple case when all random variables are Gaussian and show empirically that the algorithm converges faster for models that are inherently more forest-like.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T A Forest Mixture Bound for Block-Free Parallel Inference %A Neal Lawton %A Greg Ver Steeg %A Aram Galstyan %B Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2018 %E Amir Globerson %E Ricardo Silva %F pmlr-vR16-lawton18a %I PMLR %P 967--976 %U https://proceedings.mlr.press/r16/lawton18a.html %V R16 %X Coordinate ascent variational inference is an important algorithm for inference in proba- bilistic models, but it is slow because it updates only a single variable at a time. Block coordi- nate methods perform inference faster by up- dating blocks of variables in parallel. How- ever, the speed and convergence of these algo- rithms depends on how the variables are par- titioned into blocks. In this paper, we give a convergent parallel algorithm for inference in deep exponential families that doesn’t require the variables to be partitioned into blocks. We achieve this by lower bounding the ELBO by a new objective we call the forest mixture bound (FM bound) that separates the inference prob- lem for variables within a hidden layer. We apply this to the simple case when all random variables are Gaussian and show empirically that the algorithm converges faster for models that are inherently more forest-like. %Z Reissued by PMLR on 04 October 2026.
APA
Lawton, N., Steeg, G.V. & Galstyan, A.. (2018). A Forest Mixture Bound for Block-Free Parallel Inference. Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R16:967-976 Available from https://proceedings.mlr.press/r16/lawton18a.html. Reissued by PMLR on 04 October 2026.

Related Material