Discriminative Training of Sum-Product Networks by Extended Baum-Welch

Abdullah Rashwan, Pascal Poupart, Chen Zhitang
Proceedings of the Ninth International Conference on Probabilistic Graphical Models, PMLR 72:356-367, 2018.

Abstract

We present a discriminative learning algorithm for Sum-Product Networks (SPNs) \citep{poon2011sum} based on the Extended Baum-Welch (EBW) algorithm \citep{baum1970maximization}. We formulate the conditional data likelihood in the SPN framework as a rational function, and we use EBW to monotonically maximize it. We derive the algorithm for SPNs with both discrete and continuous variables. The experiments show that this algorithm performs better than both generative Expectation-Maximization, and discriminative gradient descent on a wide variety of applications. We also demonstrate the robustness of the algorithm in the case of missing features by comparing its performance to Support Vector Machines and Neural Networks.

Cite this Paper

BibTeX
@InProceedings{pmlr-v72-rashwan18a, title = {Discriminative Training of Sum-Product Networks by Extended Baum-Welch}, author = {Rashwan, Abdullah and Poupart, Pascal and Zhitang, Chen}, booktitle = {Proceedings of the Ninth International Conference on Probabilistic Graphical Models}, pages = {356--367}, year = {2018}, editor = {Kratochvíl, Václav and Studený, Milan}, volume = {72}, series = {Proceedings of Machine Learning Research}, month = {11--14 Sep}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v72/rashwan18a/rashwan18a.pdf}, url = {https://proceedings.mlr.press/v72/rashwan18a.html}, abstract = {We present a discriminative learning algorithm for Sum-Product Networks (SPNs) \citep{poon2011sum} based on the Extended Baum-Welch (EBW) algorithm \citep{baum1970maximization}. We formulate the conditional data likelihood in the SPN framework as a rational function, and we use EBW to monotonically maximize it. We derive the algorithm for SPNs with both discrete and continuous variables. The experiments show that this algorithm performs better than both generative Expectation-Maximization, and discriminative gradient descent on a wide variety of applications. We also demonstrate the robustness of the algorithm in the case of missing features by comparing its performance to Support Vector Machines and Neural Networks.} }
Endnote
%0 Conference Paper %T Discriminative Training of Sum-Product Networks by Extended Baum-Welch %A Abdullah Rashwan %A Pascal Poupart %A Chen Zhitang %B Proceedings of the Ninth International Conference on Probabilistic Graphical Models %C Proceedings of Machine Learning Research %D 2018 %E Václav Kratochvíl %E Milan Studený %F pmlr-v72-rashwan18a %I PMLR %P 356--367 %U https://proceedings.mlr.press/v72/rashwan18a.html %V 72 %X We present a discriminative learning algorithm for Sum-Product Networks (SPNs) \citep{poon2011sum} based on the Extended Baum-Welch (EBW) algorithm \citep{baum1970maximization}. We formulate the conditional data likelihood in the SPN framework as a rational function, and we use EBW to monotonically maximize it. We derive the algorithm for SPNs with both discrete and continuous variables. The experiments show that this algorithm performs better than both generative Expectation-Maximization, and discriminative gradient descent on a wide variety of applications. We also demonstrate the robustness of the algorithm in the case of missing features by comparing its performance to Support Vector Machines and Neural Networks.
APA
Rashwan, A., Poupart, P. & Zhitang, C.. (2018). Discriminative Training of Sum-Product Networks by Extended Baum-Welch. Proceedings of the Ninth International Conference on Probabilistic Graphical Models, in Proceedings of Machine Learning Research 72:356-367 Available from https://proceedings.mlr.press/v72/rashwan18a.html.

Related Material