Structural Expectation Propagation (SEP): Bayesian structure learning for networks with latent variables

Nevena Lazic, Christopher Bishop, John Winn
Proceedings of the Sixteenth International Conference on Artificial Intelligence and Statistics, PMLR 31:379-387, 2013.

Abstract

Learning the structure of discrete Bayesian networks has been the subject of extensive research in machine learning, with most Bayesian approaches focusing on fully observed networks. One of few the methods that can handle networks with latent variables is the "structural EM algorithm" which interleaves greedy structure search with the estimation of latent variables and parameters, maintaining a single best network at each step. We introduce Structural Expectation Propagation (SEP), an extension of EP which can infer the structure of Bayesian networks having latent variables and missing data. SEP performs variational inference in a joint model of structure, latent variables, and parameters, offering two advantages: (i) it accounts for uncertainty in structure and parameter values when making local distribution updates (ii) it returns a variational distribution over network structures rather than a single network. We demonstrate the performance of SEP both on synthetic problems and on real-world clinical data.

Cite this Paper

BibTeX
@InProceedings{pmlr-v31-lazic13a, title = {Structural Expectation Propagation (SEP): Bayesian structure learning for networks with latent variables}, author = {Lazic, Nevena and Bishop, Christopher and Winn, John}, booktitle = {Proceedings of the Sixteenth International Conference on Artificial Intelligence and Statistics}, pages = {379--387}, year = {2013}, editor = {Carvalho, Carlos M. and Ravikumar, Pradeep}, volume = {31}, series = {Proceedings of Machine Learning Research}, address = {Scottsdale, Arizona, USA}, month = {29 Apr--01 May}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v31/lazic13a.pdf}, url = {https://proceedings.mlr.press/v31/lazic13a.html}, abstract = {Learning the structure of discrete Bayesian networks has been the subject of extensive research in machine learning, with most Bayesian approaches focusing on fully observed networks. One of few the methods that can handle networks with latent variables is the "structural EM algorithm" which interleaves greedy structure search with the estimation of latent variables and parameters, maintaining a single best network at each step. We introduce Structural Expectation Propagation (SEP), an extension of EP which can infer the structure of Bayesian networks having latent variables and missing data. SEP performs variational inference in a joint model of structure, latent variables, and parameters, offering two advantages: (i) it accounts for uncertainty in structure and parameter values when making local distribution updates (ii) it returns a variational distribution over network structures rather than a single network. We demonstrate the performance of SEP both on synthetic problems and on real-world clinical data. } }
Endnote
%0 Conference Paper %T Structural Expectation Propagation (SEP): Bayesian structure learning for networks with latent variables %A Nevena Lazic %A Christopher Bishop %A John Winn %B Proceedings of the Sixteenth International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2013 %E Carlos M. Carvalho %E Pradeep Ravikumar %F pmlr-v31-lazic13a %I PMLR %P 379--387 %U https://proceedings.mlr.press/v31/lazic13a.html %V 31 %X Learning the structure of discrete Bayesian networks has been the subject of extensive research in machine learning, with most Bayesian approaches focusing on fully observed networks. One of few the methods that can handle networks with latent variables is the "structural EM algorithm" which interleaves greedy structure search with the estimation of latent variables and parameters, maintaining a single best network at each step. We introduce Structural Expectation Propagation (SEP), an extension of EP which can infer the structure of Bayesian networks having latent variables and missing data. SEP performs variational inference in a joint model of structure, latent variables, and parameters, offering two advantages: (i) it accounts for uncertainty in structure and parameter values when making local distribution updates (ii) it returns a variational distribution over network structures rather than a single network. We demonstrate the performance of SEP both on synthetic problems and on real-world clinical data.
RIS
TY - CPAPER TI - Structural Expectation Propagation (SEP): Bayesian structure learning for networks with latent variables AU - Nevena Lazic AU - Christopher Bishop AU - John Winn BT - Proceedings of the Sixteenth International Conference on Artificial Intelligence and Statistics DA - 2013/04/29 ED - Carlos M. Carvalho ED - Pradeep Ravikumar ID - pmlr-v31-lazic13a PB - PMLR DP - Proceedings of Machine Learning Research VL - 31 SP - 379 EP - 387 L1 - http://proceedings.mlr.press/v31/lazic13a.pdf UR - https://proceedings.mlr.press/v31/lazic13a.html AB - Learning the structure of discrete Bayesian networks has been the subject of extensive research in machine learning, with most Bayesian approaches focusing on fully observed networks. One of few the methods that can handle networks with latent variables is the "structural EM algorithm" which interleaves greedy structure search with the estimation of latent variables and parameters, maintaining a single best network at each step. We introduce Structural Expectation Propagation (SEP), an extension of EP which can infer the structure of Bayesian networks having latent variables and missing data. SEP performs variational inference in a joint model of structure, latent variables, and parameters, offering two advantages: (i) it accounts for uncertainty in structure and parameter values when making local distribution updates (ii) it returns a variational distribution over network structures rather than a single network. We demonstrate the performance of SEP both on synthetic problems and on real-world clinical data. ER -
APA
Lazic, N., Bishop, C. & Winn, J.. (2013). Structural Expectation Propagation (SEP): Bayesian structure learning for networks with latent variables. Proceedings of the Sixteenth International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 31:379-387 Available from https://proceedings.mlr.press/v31/lazic13a.html.

Related Material