Clamping Improves TRW and Mean Field Approximations

Adrian Weller, Justin Domke
Proceedings of the 19th International Conference on Artificial Intelligence and Statistics, PMLR 51:38-46, 2016.

Abstract

We examine the effect of clamping variables for approximate inference in undirected graphical models with pairwise relationships and discrete variables. For any number of variable labels, we demonstrate that clamping and summing approximate sub-partition functions can lead only to a decrease in the partition function estimate for TRW, and an increase for the naive mean field method, in each case guaranteeing an improvement in the approximation and bound. We next focus on binary variables, add the Bethe approximation to consideration and examine ways to choose good variables to clamp, introducing new methods. We show the importance of identifying highly frustrated cycles, and of checking the singleton entropy of a variable. We explore the value of our methods by empirical analysis and draw lessons to guide practitioners.

Cite this Paper

BibTeX
@InProceedings{pmlr-v51-weller16a, title = {Clamping Improves TRW and Mean Field Approximations}, author = {Weller, Adrian and Domke, Justin}, booktitle = {Proceedings of the 19th International Conference on Artificial Intelligence and Statistics}, pages = {38--46}, year = {2016}, editor = {Gretton, Arthur and Robert, Christian C.}, volume = {51}, series = {Proceedings of Machine Learning Research}, address = {Cadiz, Spain}, month = {09--11 May}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v51/weller16a.pdf}, url = {https://proceedings.mlr.press/v51/weller16a.html}, abstract = {We examine the effect of clamping variables for approximate inference in undirected graphical models with pairwise relationships and discrete variables. For any number of variable labels, we demonstrate that clamping and summing approximate sub-partition functions can lead only to a decrease in the partition function estimate for TRW, and an increase for the naive mean field method, in each case guaranteeing an improvement in the approximation and bound. We next focus on binary variables, add the Bethe approximation to consideration and examine ways to choose good variables to clamp, introducing new methods. We show the importance of identifying highly frustrated cycles, and of checking the singleton entropy of a variable. We explore the value of our methods by empirical analysis and draw lessons to guide practitioners.} }
Endnote
%0 Conference Paper %T Clamping Improves TRW and Mean Field Approximations %A Adrian Weller %A Justin Domke %B Proceedings of the 19th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2016 %E Arthur Gretton %E Christian C. Robert %F pmlr-v51-weller16a %I PMLR %P 38--46 %U https://proceedings.mlr.press/v51/weller16a.html %V 51 %X We examine the effect of clamping variables for approximate inference in undirected graphical models with pairwise relationships and discrete variables. For any number of variable labels, we demonstrate that clamping and summing approximate sub-partition functions can lead only to a decrease in the partition function estimate for TRW, and an increase for the naive mean field method, in each case guaranteeing an improvement in the approximation and bound. We next focus on binary variables, add the Bethe approximation to consideration and examine ways to choose good variables to clamp, introducing new methods. We show the importance of identifying highly frustrated cycles, and of checking the singleton entropy of a variable. We explore the value of our methods by empirical analysis and draw lessons to guide practitioners.
RIS
TY - CPAPER TI - Clamping Improves TRW and Mean Field Approximations AU - Adrian Weller AU - Justin Domke BT - Proceedings of the 19th International Conference on Artificial Intelligence and Statistics DA - 2016/05/02 ED - Arthur Gretton ED - Christian C. Robert ID - pmlr-v51-weller16a PB - PMLR DP - Proceedings of Machine Learning Research VL - 51 SP - 38 EP - 46 L1 - http://proceedings.mlr.press/v51/weller16a.pdf UR - https://proceedings.mlr.press/v51/weller16a.html AB - We examine the effect of clamping variables for approximate inference in undirected graphical models with pairwise relationships and discrete variables. For any number of variable labels, we demonstrate that clamping and summing approximate sub-partition functions can lead only to a decrease in the partition function estimate for TRW, and an increase for the naive mean field method, in each case guaranteeing an improvement in the approximation and bound. We next focus on binary variables, add the Bethe approximation to consideration and examine ways to choose good variables to clamp, introducing new methods. We show the importance of identifying highly frustrated cycles, and of checking the singleton entropy of a variable. We explore the value of our methods by empirical analysis and draw lessons to guide practitioners. ER -
APA
Weller, A. & Domke, J.. (2016). Clamping Improves TRW and Mean Field Approximations. Proceedings of the 19th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 51:38-46 Available from https://proceedings.mlr.press/v51/weller16a.html.

Related Material