Locally Minimax Optimal Predictive Modeling with Bayesian Networks

Tomi Silander, Teemu Roos, Petri Myllymäki
; Proceedings of the Twelth International Conference on Artificial Intelligence and Statistics, PMLR 5:504-511, 2009.

Abstract

We propose an information-theoretic approach for predictive modeling with Bayesian networks. Our approach is based on the minimax optimal Normalized Maximum Likelihood (NML) distribution, motivated by the MDL principle. In particular, we present a parameter learning method which, together with a previously introduced NML-based model selection criterion, provides a way to construct highly predictive Bayesian network models from data. The method is parameter-free and robust, unlike the currently popular Bayesian marginal likelihood approach which has been shown to be sensitive to the choice of prior hyperparameters. Empirical tests show that the proposed method compares favorably with the Bayesian approach in predictive tasks.

Cite this Paper


BibTeX
@InProceedings{pmlr-v5-silander09a, title = {Locally Minimax Optimal Predictive Modeling with Bayesian Networks}, author = {Tomi Silander and Teemu Roos and Petri Myllymäki}, booktitle = {Proceedings of the Twelth International Conference on Artificial Intelligence and Statistics}, pages = {504--511}, year = {2009}, editor = {David van Dyk and Max Welling}, volume = {5}, series = {Proceedings of Machine Learning Research}, address = {Hilton Clearwater Beach Resort, Clearwater Beach, Florida USA}, month = {16--18 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v5/silander09a/silander09a.pdf}, url = {http://proceedings.mlr.press/v5/silander09a.html}, abstract = {We propose an information-theoretic approach for predictive modeling with Bayesian networks. Our approach is based on the minimax optimal Normalized Maximum Likelihood (NML) distribution, motivated by the MDL principle. In particular, we present a parameter learning method which, together with a previously introduced NML-based model selection criterion, provides a way to construct highly predictive Bayesian network models from data. The method is parameter-free and robust, unlike the currently popular Bayesian marginal likelihood approach which has been shown to be sensitive to the choice of prior hyperparameters. Empirical tests show that the proposed method compares favorably with the Bayesian approach in predictive tasks.} }
Endnote
%0 Conference Paper %T Locally Minimax Optimal Predictive Modeling with Bayesian Networks %A Tomi Silander %A Teemu Roos %A Petri Myllymäki %B Proceedings of the Twelth International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2009 %E David van Dyk %E Max Welling %F pmlr-v5-silander09a %I PMLR %J Proceedings of Machine Learning Research %P 504--511 %U http://proceedings.mlr.press %V 5 %W PMLR %X We propose an information-theoretic approach for predictive modeling with Bayesian networks. Our approach is based on the minimax optimal Normalized Maximum Likelihood (NML) distribution, motivated by the MDL principle. In particular, we present a parameter learning method which, together with a previously introduced NML-based model selection criterion, provides a way to construct highly predictive Bayesian network models from data. The method is parameter-free and robust, unlike the currently popular Bayesian marginal likelihood approach which has been shown to be sensitive to the choice of prior hyperparameters. Empirical tests show that the proposed method compares favorably with the Bayesian approach in predictive tasks.
RIS
TY - CPAPER TI - Locally Minimax Optimal Predictive Modeling with Bayesian Networks AU - Tomi Silander AU - Teemu Roos AU - Petri Myllymäki BT - Proceedings of the Twelth International Conference on Artificial Intelligence and Statistics PY - 2009/04/15 DA - 2009/04/15 ED - David van Dyk ED - Max Welling ID - pmlr-v5-silander09a PB - PMLR SP - 504 DP - PMLR EP - 511 L1 - http://proceedings.mlr.press/v5/silander09a/silander09a.pdf UR - http://proceedings.mlr.press/v5/silander09a.html AB - We propose an information-theoretic approach for predictive modeling with Bayesian networks. Our approach is based on the minimax optimal Normalized Maximum Likelihood (NML) distribution, motivated by the MDL principle. In particular, we present a parameter learning method which, together with a previously introduced NML-based model selection criterion, provides a way to construct highly predictive Bayesian network models from data. The method is parameter-free and robust, unlike the currently popular Bayesian marginal likelihood approach which has been shown to be sensitive to the choice of prior hyperparameters. Empirical tests show that the proposed method compares favorably with the Bayesian approach in predictive tasks. ER -
APA
Silander, T., Roos, T. & Myllymäki, P.. (2009). Locally Minimax Optimal Predictive Modeling with Bayesian Networks. Proceedings of the Twelth International Conference on Artificial Intelligence and Statistics, in PMLR 5:504-511

Related Material