Optimization of Structured Mean Field Objectives

Alexandre Bouchard-Côté, Mike Jordan
Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence, PMLR R7:59-66, 2009.

Abstract

In intractable, undirected graphical models, an intuitive way of creating structured mean field approximations is to select an acyclic tractable subgraph. We show that the hardness of computing the objective function and gradient of the mean field objective qualitatively depends on a simple graph property. If the tractable subgraph has this property- we call such subgraphs v-acyclic-a very fast block coordinate ascent algorithm is possible. If not, optimization is harder, but we show a new algorithm based on the construction of an auxiliary exponential family that can be used to make inference possible in this case as well. We discuss the advantages and disadvantages of each regime and compare the algorithms empirically.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR7-bouchard-cote09a, title = {Optimization of Structured Mean Field Objectives}, author = {Bouchard-C{\^o}t{\'e}, Alexandre and Jordan, Mike}, booktitle = {Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence}, pages = {59--66}, year = {2009}, editor = {Bilmes, Jeff and Ng, Andrew Y.}, volume = {R7}, series = {Proceedings of Machine Learning Research}, month = {18--21 Jun}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r7/main/assets/bouchard-cote09a/bouchard-cote09a.pdf}, url = {https://proceedings.mlr.press/r7/bouchard-cote09a.html}, abstract = {In intractable, undirected graphical models, an intuitive way of creating structured mean field approximations is to select an acyclic tractable subgraph. We show that the hardness of computing the objective function and gradient of the mean field objective qualitatively depends on a simple graph property. If the tractable subgraph has this property- we call such subgraphs v-acyclic-a very fast block coordinate ascent algorithm is possible. If not, optimization is harder, but we show a new algorithm based on the construction of an auxiliary exponential family that can be used to make inference possible in this case as well. We discuss the advantages and disadvantages of each regime and compare the algorithms empirically.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T Optimization of Structured Mean Field Objectives %A Alexandre Bouchard-Côté %A Mike Jordan %B Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2009 %E Jeff Bilmes %E Andrew Y. Ng %F pmlr-vR7-bouchard-cote09a %I PMLR %P 59--66 %U https://proceedings.mlr.press/r7/bouchard-cote09a.html %V R7 %X In intractable, undirected graphical models, an intuitive way of creating structured mean field approximations is to select an acyclic tractable subgraph. We show that the hardness of computing the objective function and gradient of the mean field objective qualitatively depends on a simple graph property. If the tractable subgraph has this property- we call such subgraphs v-acyclic-a very fast block coordinate ascent algorithm is possible. If not, optimization is harder, but we show a new algorithm based on the construction of an auxiliary exponential family that can be used to make inference possible in this case as well. We discuss the advantages and disadvantages of each regime and compare the algorithms empirically. %Z Reissued by PMLR on 04 October 2026.
APA
Bouchard-Côté, A. & Jordan, M.. (2009). Optimization of Structured Mean Field Objectives. Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R7:59-66 Available from https://proceedings.mlr.press/r7/bouchard-cote09a.html. Reissued by PMLR on 04 October 2026.

Related Material