Informative Priors for Markov Blanket Discovery

Adam Pocock, Mikel Lujan, Gavin Brown
; Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, PMLR 22:905-913, 2012.

Abstract

We present a novel interpretation of information theoretic feature selection as optimization of a discriminative model. We show that this formulation coincides with a group of mutual information based filter heuristics in the literature, and show how our probabilistic framework gives a well-founded extension for informative priors. We then derive a particular sparsity prior that recovers the well-known IAMB algorithm (Tsamardinos & Aliferis, 2003) and extend it to create a novel algorithm, IAMB-IP, that includes domain knowledge priors. In empirical evaluations, we find the new algorithm to improve Markov Blanket recovery even when a misspecified prior was used, in which half the prior knowledge was incorrect.

Cite this Paper


BibTeX
@InProceedings{pmlr-v22-pocock12, title = {Informative Priors for Markov Blanket Discovery}, author = {Adam Pocock and Mikel Lujan and Gavin Brown}, booktitle = {Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics}, pages = {905--913}, year = {2012}, editor = {Neil D. Lawrence and Mark Girolami}, volume = {22}, series = {Proceedings of Machine Learning Research}, address = {La Palma, Canary Islands}, month = {21--23 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v22/pocock12/pocock12.pdf}, url = {http://proceedings.mlr.press/v22/pocock12.html}, abstract = {We present a novel interpretation of information theoretic feature selection as optimization of a discriminative model. We show that this formulation coincides with a group of mutual information based filter heuristics in the literature, and show how our probabilistic framework gives a well-founded extension for informative priors. We then derive a particular sparsity prior that recovers the well-known IAMB algorithm (Tsamardinos & Aliferis, 2003) and extend it to create a novel algorithm, IAMB-IP, that includes domain knowledge priors. In empirical evaluations, we find the new algorithm to improve Markov Blanket recovery even when a misspecified prior was used, in which half the prior knowledge was incorrect.} }
Endnote
%0 Conference Paper %T Informative Priors for Markov Blanket Discovery %A Adam Pocock %A Mikel Lujan %A Gavin Brown %B Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2012 %E Neil D. Lawrence %E Mark Girolami %F pmlr-v22-pocock12 %I PMLR %J Proceedings of Machine Learning Research %P 905--913 %U http://proceedings.mlr.press %V 22 %W PMLR %X We present a novel interpretation of information theoretic feature selection as optimization of a discriminative model. We show that this formulation coincides with a group of mutual information based filter heuristics in the literature, and show how our probabilistic framework gives a well-founded extension for informative priors. We then derive a particular sparsity prior that recovers the well-known IAMB algorithm (Tsamardinos & Aliferis, 2003) and extend it to create a novel algorithm, IAMB-IP, that includes domain knowledge priors. In empirical evaluations, we find the new algorithm to improve Markov Blanket recovery even when a misspecified prior was used, in which half the prior knowledge was incorrect.
RIS
TY - CPAPER TI - Informative Priors for Markov Blanket Discovery AU - Adam Pocock AU - Mikel Lujan AU - Gavin Brown BT - Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics PY - 2012/03/21 DA - 2012/03/21 ED - Neil D. Lawrence ED - Mark Girolami ID - pmlr-v22-pocock12 PB - PMLR SP - 905 DP - PMLR EP - 913 L1 - http://proceedings.mlr.press/v22/pocock12/pocock12.pdf UR - http://proceedings.mlr.press/v22/pocock12.html AB - We present a novel interpretation of information theoretic feature selection as optimization of a discriminative model. We show that this formulation coincides with a group of mutual information based filter heuristics in the literature, and show how our probabilistic framework gives a well-founded extension for informative priors. We then derive a particular sparsity prior that recovers the well-known IAMB algorithm (Tsamardinos & Aliferis, 2003) and extend it to create a novel algorithm, IAMB-IP, that includes domain knowledge priors. In empirical evaluations, we find the new algorithm to improve Markov Blanket recovery even when a misspecified prior was used, in which half the prior knowledge was incorrect. ER -
APA
Pocock, A., Lujan, M. & Brown, G.. (2012). Informative Priors for Markov Blanket Discovery. Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, in PMLR 22:905-913

Related Material