Online and Distributed Bayesian Moment Matching for Parameter Learning in Sum-Product Networks

Abdullah Rashwan, Han Zhao, Pascal Poupart
Proceedings of the 19th International Conference on Artificial Intelligence and Statistics, PMLR 51:1469-1477, 2016.

Abstract

Probabilistic graphical models provide a general and flexible framework for reasoning about complex dependencies in noisy domains with many variables. Among the various types of probabilistic graphical models, sum-product networks (SPNs) have recently generated some interest because exact inference can always be done in linear time with respect to the size of the network. This is particularly attractive since it means that learning an SPN from data always yields a tractable model for inference. However, existing parameter learning algorithms for SPNs operate in batch mode and do not scale easily to large datasets. In this work, we explore online algorithms to ensure that parameter learning can also be done tractably with respect to the amount of data. More specifically, we propose a new Bayesian moment matching (BMM) algorithm that operates naturally in an online fashion and that can be easily distributed. We demonstrate the effectiveness and scalability of BMM in comparison to online extensions of gradient descent, exponentiated gradient and expectation maximization on 20 classic benchmarks and 4 large scale datasets.

Cite this Paper

BibTeX
@InProceedings{pmlr-v51-rashwan16, title = {Online and Distributed Bayesian Moment Matching for Parameter Learning in Sum-Product Networks}, author = {Rashwan, Abdullah and Zhao, Han and Poupart, Pascal}, booktitle = {Proceedings of the 19th International Conference on Artificial Intelligence and Statistics}, pages = {1469--1477}, year = {2016}, editor = {Gretton, Arthur and Robert, Christian C.}, volume = {51}, series = {Proceedings of Machine Learning Research}, address = {Cadiz, Spain}, month = {09--11 May}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v51/rashwan16.pdf}, url = {https://proceedings.mlr.press/v51/rashwan16.html}, abstract = {Probabilistic graphical models provide a general and flexible framework for reasoning about complex dependencies in noisy domains with many variables. Among the various types of probabilistic graphical models, sum-product networks (SPNs) have recently generated some interest because exact inference can always be done in linear time with respect to the size of the network. This is particularly attractive since it means that learning an SPN from data always yields a tractable model for inference. However, existing parameter learning algorithms for SPNs operate in batch mode and do not scale easily to large datasets. In this work, we explore online algorithms to ensure that parameter learning can also be done tractably with respect to the amount of data. More specifically, we propose a new Bayesian moment matching (BMM) algorithm that operates naturally in an online fashion and that can be easily distributed. We demonstrate the effectiveness and scalability of BMM in comparison to online extensions of gradient descent, exponentiated gradient and expectation maximization on 20 classic benchmarks and 4 large scale datasets.} }
Endnote
%0 Conference Paper %T Online and Distributed Bayesian Moment Matching for Parameter Learning in Sum-Product Networks %A Abdullah Rashwan %A Han Zhao %A Pascal Poupart %B Proceedings of the 19th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2016 %E Arthur Gretton %E Christian C. Robert %F pmlr-v51-rashwan16 %I PMLR %P 1469--1477 %U https://proceedings.mlr.press/v51/rashwan16.html %V 51 %X Probabilistic graphical models provide a general and flexible framework for reasoning about complex dependencies in noisy domains with many variables. Among the various types of probabilistic graphical models, sum-product networks (SPNs) have recently generated some interest because exact inference can always be done in linear time with respect to the size of the network. This is particularly attractive since it means that learning an SPN from data always yields a tractable model for inference. However, existing parameter learning algorithms for SPNs operate in batch mode and do not scale easily to large datasets. In this work, we explore online algorithms to ensure that parameter learning can also be done tractably with respect to the amount of data. More specifically, we propose a new Bayesian moment matching (BMM) algorithm that operates naturally in an online fashion and that can be easily distributed. We demonstrate the effectiveness and scalability of BMM in comparison to online extensions of gradient descent, exponentiated gradient and expectation maximization on 20 classic benchmarks and 4 large scale datasets.
RIS
TY - CPAPER TI - Online and Distributed Bayesian Moment Matching for Parameter Learning in Sum-Product Networks AU - Abdullah Rashwan AU - Han Zhao AU - Pascal Poupart BT - Proceedings of the 19th International Conference on Artificial Intelligence and Statistics DA - 2016/05/02 ED - Arthur Gretton ED - Christian C. Robert ID - pmlr-v51-rashwan16 PB - PMLR DP - Proceedings of Machine Learning Research VL - 51 SP - 1469 EP - 1477 L1 - http://proceedings.mlr.press/v51/rashwan16.pdf UR - https://proceedings.mlr.press/v51/rashwan16.html AB - Probabilistic graphical models provide a general and flexible framework for reasoning about complex dependencies in noisy domains with many variables. Among the various types of probabilistic graphical models, sum-product networks (SPNs) have recently generated some interest because exact inference can always be done in linear time with respect to the size of the network. This is particularly attractive since it means that learning an SPN from data always yields a tractable model for inference. However, existing parameter learning algorithms for SPNs operate in batch mode and do not scale easily to large datasets. In this work, we explore online algorithms to ensure that parameter learning can also be done tractably with respect to the amount of data. More specifically, we propose a new Bayesian moment matching (BMM) algorithm that operates naturally in an online fashion and that can be easily distributed. We demonstrate the effectiveness and scalability of BMM in comparison to online extensions of gradient descent, exponentiated gradient and expectation maximization on 20 classic benchmarks and 4 large scale datasets. ER -
APA
Rashwan, A., Zhao, H. & Poupart, P.. (2016). Online and Distributed Bayesian Moment Matching for Parameter Learning in Sum-Product Networks. Proceedings of the 19th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 51:1469-1477 Available from https://proceedings.mlr.press/v51/rashwan16.html.

Related Material