Bayesian Structure Learning for Markov Random Fields with a Spike and Slab Prior

Yutian Chen, Max Welling
Proceedings of the 28th Conference on Uncertainty in Artificial Intelligence, PMLR R10:172-182, 2012.

Abstract

In recent years a number of methods have been developed for automatically learning the (sparse) connectivity structure of Markov Random Fields. These methods are mostly based on L1-regularized optimization which has a number of disadvantages such as the inability to assess model uncertainty and expensive crossvalidation to find the optimal regularization parameter. Moreover, the model’s predictive performance may degrade dramatically with a suboptimal value of the regularization parameter (which is sometimes desirable to induce sparseness). We propose a fully Bayesian approach based on a "spike and slab" prior (similar to L0 regularization) that does not suffer from these shortcomings. We develop an approximate MCMC method combining Langevin dynamics and reversible jump MCMC to conduct inference in this model. Experiments show that the proposed model learns a good combination of the structure and parameter values without the need for separate hyper-parameter tuning. Moreover, the model’s predictive performance is much more robust than L1-based methods with hyper-parameter settings that induce highly sparse model structures.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR10-chen12b, title = {{B}ayesian Structure Learning for {M}arkov Random Fields with a Spike and Slab Prior}, author = {Chen, Yutian and Welling, Max}, booktitle = {Proceedings of the 28th Conference on Uncertainty in Artificial Intelligence}, pages = {172--182}, year = {2012}, editor = {de Freitas, Nando and Murphy, Kevin}, volume = {R10}, series = {Proceedings of Machine Learning Research}, month = {14--18 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r10/main/assets/chen12b/chen12b.pdf}, url = {https://proceedings.mlr.press/r10/chen12b.html}, abstract = {In recent years a number of methods have been developed for automatically learning the (sparse) connectivity structure of Markov Random Fields. These methods are mostly based on L1-regularized optimization which has a number of disadvantages such as the inability to assess model uncertainty and expensive crossvalidation to find the optimal regularization parameter. Moreover, the model’s predictive performance may degrade dramatically with a suboptimal value of the regularization parameter (which is sometimes desirable to induce sparseness). We propose a fully Bayesian approach based on a "spike and slab" prior (similar to L0 regularization) that does not suffer from these shortcomings. We develop an approximate MCMC method combining Langevin dynamics and reversible jump MCMC to conduct inference in this model. Experiments show that the proposed model learns a good combination of the structure and parameter values without the need for separate hyper-parameter tuning. Moreover, the model’s predictive performance is much more robust than L1-based methods with hyper-parameter settings that induce highly sparse model structures.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T Bayesian Structure Learning for Markov Random Fields with a Spike and Slab Prior %A Yutian Chen %A Max Welling %B Proceedings of the 28th Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2012 %E Nando de Freitas %E Kevin Murphy %F pmlr-vR10-chen12b %I PMLR %P 172--182 %U https://proceedings.mlr.press/r10/chen12b.html %V R10 %X In recent years a number of methods have been developed for automatically learning the (sparse) connectivity structure of Markov Random Fields. These methods are mostly based on L1-regularized optimization which has a number of disadvantages such as the inability to assess model uncertainty and expensive crossvalidation to find the optimal regularization parameter. Moreover, the model’s predictive performance may degrade dramatically with a suboptimal value of the regularization parameter (which is sometimes desirable to induce sparseness). We propose a fully Bayesian approach based on a "spike and slab" prior (similar to L0 regularization) that does not suffer from these shortcomings. We develop an approximate MCMC method combining Langevin dynamics and reversible jump MCMC to conduct inference in this model. Experiments show that the proposed model learns a good combination of the structure and parameter values without the need for separate hyper-parameter tuning. Moreover, the model’s predictive performance is much more robust than L1-based methods with hyper-parameter settings that induce highly sparse model structures. %Z Reissued by PMLR on 04 October 2026.
APA
Chen, Y. & Welling, M.. (2012). Bayesian Structure Learning for Markov Random Fields with a Spike and Slab Prior. Proceedings of the 28th Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R10:172-182 Available from https://proceedings.mlr.press/r10/chen12b.html. Reissued by PMLR on 04 October 2026.

Related Material