Informative Priors for Markov Blanket Discovery

Adam Pocock, Mikel Lujan, Gavin Brown
Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, PMLR 22:905-913, 2012.

Abstract

We present a novel interpretation of information theoretic feature selection as optimization of a discriminative model. We show that this formulation coincides with a group of mutual information based filter heuristics in the literature, and show how our probabilistic framework gives a well-founded extension for informative priors. We then derive a particular sparsity prior that recovers the well-known IAMB algorithm (Tsamardinos & Aliferis, 2003) and extend it to create a novel algorithm, IAMB-IP, that includes domain knowledge priors. In empirical evaluations, we find the new algorithm to improve Markov Blanket recovery even when a misspecified prior was used, in which half the prior knowledge was incorrect.

Cite this Paper

BibTeX
@InProceedings{pmlr-v22-pocock12, title = {Informative Priors for Markov Blanket Discovery}, author = {Pocock, Adam and Lujan, Mikel and Brown, Gavin}, booktitle = {Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics}, pages = {905--913}, year = {2012}, editor = {Lawrence, Neil D. and Girolami, Mark}, volume = {22}, series = {Proceedings of Machine Learning Research}, address = {La Palma, Canary Islands}, month = {21--23 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v22/pocock12/pocock12.pdf}, url = {https://proceedings.mlr.press/v22/pocock12.html}, abstract = {We present a novel interpretation of information theoretic feature selection as optimization of a discriminative model. We show that this formulation coincides with a group of mutual information based filter heuristics in the literature, and show how our probabilistic framework gives a well-founded extension for informative priors. We then derive a particular sparsity prior that recovers the well-known IAMB algorithm (Tsamardinos & Aliferis, 2003) and extend it to create a novel algorithm, IAMB-IP, that includes domain knowledge priors. In empirical evaluations, we find the new algorithm to improve Markov Blanket recovery even when a misspecified prior was used, in which half the prior knowledge was incorrect.} }
Endnote
%0 Conference Paper %T Informative Priors for Markov Blanket Discovery %A Adam Pocock %A Mikel Lujan %A Gavin Brown %B Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2012 %E Neil D. Lawrence %E Mark Girolami %F pmlr-v22-pocock12 %I PMLR %P 905--913 %U https://proceedings.mlr.press/v22/pocock12.html %V 22 %X We present a novel interpretation of information theoretic feature selection as optimization of a discriminative model. We show that this formulation coincides with a group of mutual information based filter heuristics in the literature, and show how our probabilistic framework gives a well-founded extension for informative priors. We then derive a particular sparsity prior that recovers the well-known IAMB algorithm (Tsamardinos & Aliferis, 2003) and extend it to create a novel algorithm, IAMB-IP, that includes domain knowledge priors. In empirical evaluations, we find the new algorithm to improve Markov Blanket recovery even when a misspecified prior was used, in which half the prior knowledge was incorrect.
RIS
TY - CPAPER TI - Informative Priors for Markov Blanket Discovery AU - Adam Pocock AU - Mikel Lujan AU - Gavin Brown BT - Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics DA - 2012/03/21 ED - Neil D. Lawrence ED - Mark Girolami ID - pmlr-v22-pocock12 PB - PMLR DP - Proceedings of Machine Learning Research VL - 22 SP - 905 EP - 913 L1 - http://proceedings.mlr.press/v22/pocock12/pocock12.pdf UR - https://proceedings.mlr.press/v22/pocock12.html AB - We present a novel interpretation of information theoretic feature selection as optimization of a discriminative model. We show that this formulation coincides with a group of mutual information based filter heuristics in the literature, and show how our probabilistic framework gives a well-founded extension for informative priors. We then derive a particular sparsity prior that recovers the well-known IAMB algorithm (Tsamardinos & Aliferis, 2003) and extend it to create a novel algorithm, IAMB-IP, that includes domain knowledge priors. In empirical evaluations, we find the new algorithm to improve Markov Blanket recovery even when a misspecified prior was used, in which half the prior knowledge was incorrect. ER -
APA
Pocock, A., Lujan, M. & Brown, G.. (2012). Informative Priors for Markov Blanket Discovery. Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 22:905-913 Available from https://proceedings.mlr.press/v22/pocock12.html.

Related Material