Bounds on the Bethe free energy for Gaussian networks

Botond Cseke, Tom Heskes
Proceedings of the 24th Conference on Uncertainty in Artificial Intelligence, PMLR R6:97-104, 2008.

Abstract

We address the problem of computing approximate marginals in Gaussian probabilistic models by using mean field and fractional Bethe approximations. As an extension of Welling and Teh (2001), we define the Gaussian fractional Bethe free energy in terms of the moment parameters of the approximate marginals and derive an upper and lower bound for it. We give necessary conditions for the Gaussian fractional Bethe free energies to be bounded from below. It turns out that the bounding condition is the same as the pairwise normalizability condition derived by Malioutov et al. (2006) as a sufficient condition for the convergence of the message passing algorithm. By giving a counterexample, we disprove the conjecture in Welling and Teh (2001): even when the Bethe free energy is not bounded from below, it can possess a local minimum to which the minimization algorithms can converge.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR6-cseke08a, title = {Bounds on the Bethe free energy for Gaussian networks}, author = {Cseke, Botond and Heskes, Tom}, booktitle = {Proceedings of the 24th Conference on Uncertainty in Artificial Intelligence}, pages = {97--104}, year = {2008}, editor = {McAllester, David A. and Myllymäki, Petri}, volume = {R6}, series = {Proceedings of Machine Learning Research}, month = {09--12 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r6/main/assets/cseke08a/cseke08a.pdf}, url = {https://proceedings.mlr.press/r6/cseke08a.html}, abstract = {We address the problem of computing approximate marginals in Gaussian probabilistic models by using mean field and fractional Bethe approximations. As an extension of Welling and Teh (2001), we define the Gaussian fractional Bethe free energy in terms of the moment parameters of the approximate marginals and derive an upper and lower bound for it. We give necessary conditions for the Gaussian fractional Bethe free energies to be bounded from below. It turns out that the bounding condition is the same as the pairwise normalizability condition derived by Malioutov et al. (2006) as a sufficient condition for the convergence of the message passing algorithm. By giving a counterexample, we disprove the conjecture in Welling and Teh (2001): even when the Bethe free energy is not bounded from below, it can possess a local minimum to which the minimization algorithms can converge.}, note = {Reissued by PMLR on 09 October 2024.} }
Endnote
%0 Conference Paper %T Bounds on the Bethe free energy for Gaussian networks %A Botond Cseke %A Tom Heskes %B Proceedings of the 24th Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2008 %E David A. McAllester %E Petri Myllymäki %F pmlr-vR6-cseke08a %I PMLR %P 97--104 %U https://proceedings.mlr.press/r6/cseke08a.html %V R6 %X We address the problem of computing approximate marginals in Gaussian probabilistic models by using mean field and fractional Bethe approximations. As an extension of Welling and Teh (2001), we define the Gaussian fractional Bethe free energy in terms of the moment parameters of the approximate marginals and derive an upper and lower bound for it. We give necessary conditions for the Gaussian fractional Bethe free energies to be bounded from below. It turns out that the bounding condition is the same as the pairwise normalizability condition derived by Malioutov et al. (2006) as a sufficient condition for the convergence of the message passing algorithm. By giving a counterexample, we disprove the conjecture in Welling and Teh (2001): even when the Bethe free energy is not bounded from below, it can possess a local minimum to which the minimization algorithms can converge. %Z Reissued by PMLR on 09 October 2024.
APA
Cseke, B. & Heskes, T.. (2008). Bounds on the Bethe free energy for Gaussian networks. Proceedings of the 24th Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R6:97-104 Available from https://proceedings.mlr.press/r6/cseke08a.html. Reissued by PMLR on 09 October 2024.

Related Material