Differentiable TAN Structure Learning for Bayesian Network Classifiers

Wolfgang Roth, Franz Pernkopf
Proceedings of the 10th International Conference on Probabilistic Graphical Models, PMLR 138:389-400, 2020.

Abstract

Learning the structure of Bayesian networks is a difficult combinatorial optimization problem. In this paper, we consider learning of tree-augmented naive Bayes (TAN) structures for Bayesian network classifiers with discrete input features. Instead of performing a combinatorial optimization over the space of possible graph structures, the proposed method learns a distribution over graph structures. After training, we select the most probable structure of this distribution. This allows for a joint training of the Bayesian network parameters along with its TAN structure using gradient-based optimization. The proposed method is agnostic to the specific loss and only requires that it is differentiable. We perform extensive experiments using a hybrid generative-discriminative loss based on the discriminative probabilistic margin. Our method consistently outperforms random TAN structures and Chow-Liu TAN structures.

Cite this Paper

BibTeX
@InProceedings{pmlr-v138-roth20a, title = {Differentiable TAN Structure Learning for Bayesian Network Classifiers}, author = {Roth, Wolfgang and Pernkopf, Franz}, booktitle = {Proceedings of the 10th International Conference on Probabilistic Graphical Models}, pages = {389--400}, year = {2020}, editor = {Jaeger, Manfred and Nielsen, Thomas Dyhre}, volume = {138}, series = {Proceedings of Machine Learning Research}, month = {23--25 Sep}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v138/roth20a/roth20a.pdf}, url = {https://proceedings.mlr.press/v138/roth20a.html}, abstract = {Learning the structure of Bayesian networks is a difficult combinatorial optimization problem. In this paper, we consider learning of tree-augmented naive Bayes (TAN) structures for Bayesian network classifiers with discrete input features. Instead of performing a combinatorial optimization over the space of possible graph structures, the proposed method learns a distribution over graph structures. After training, we select the most probable structure of this distribution. This allows for a joint training of the Bayesian network parameters along with its TAN structure using gradient-based optimization. The proposed method is agnostic to the specific loss and only requires that it is differentiable. We perform extensive experiments using a hybrid generative-discriminative loss based on the discriminative probabilistic margin. Our method consistently outperforms random TAN structures and Chow-Liu TAN structures.} }
Endnote
%0 Conference Paper %T Differentiable TAN Structure Learning for Bayesian Network Classifiers %A Wolfgang Roth %A Franz Pernkopf %B Proceedings of the 10th International Conference on Probabilistic Graphical Models %C Proceedings of Machine Learning Research %D 2020 %E Manfred Jaeger %E Thomas Dyhre Nielsen %F pmlr-v138-roth20a %I PMLR %P 389--400 %U https://proceedings.mlr.press/v138/roth20a.html %V 138 %X Learning the structure of Bayesian networks is a difficult combinatorial optimization problem. In this paper, we consider learning of tree-augmented naive Bayes (TAN) structures for Bayesian network classifiers with discrete input features. Instead of performing a combinatorial optimization over the space of possible graph structures, the proposed method learns a distribution over graph structures. After training, we select the most probable structure of this distribution. This allows for a joint training of the Bayesian network parameters along with its TAN structure using gradient-based optimization. The proposed method is agnostic to the specific loss and only requires that it is differentiable. We perform extensive experiments using a hybrid generative-discriminative loss based on the discriminative probabilistic margin. Our method consistently outperforms random TAN structures and Chow-Liu TAN structures.
APA
Roth, W. & Pernkopf, F.. (2020). Differentiable TAN Structure Learning for Bayesian Network Classifiers. Proceedings of the 10th International Conference on Probabilistic Graphical Models, in Proceedings of Machine Learning Research 138:389-400 Available from https://proceedings.mlr.press/v138/roth20a.html.

Related Material