Learning the beta-Divergence in Tweedie Compound Poisson Matrix Factorization Models

Umut Simsekli, Ali Taylan Cemgil, Yusuf Kenan Yilmaz
Proceedings of the 30th International Conference on Machine Learning, PMLR 28(3):1409-1417, 2013.

Abstract

In this study, we derive algorithms for estimating mixed β-divergences. Such cost functions are useful for Nonnegative Matrix and Tensor Factorization models with a compound Poisson observation model. Compound Poisson is a particular Tweedie model, an important special case of exponential dispersion models characterized by the fact that the variance is proportional to a power function of the mean. There are several well known matrix and tensor factorization algorithms that minimize the β-divergence; these estimate the mean parameter. The probabilistic interpretation gives us more flexibility and robustness by providing us additional tunable parameters such as power and dispersion. Estimation of the power parameter is useful for choosing a suitable divergence and estimation of dispersion is useful for data driven regularization and weighting in collective/coupled factorization of heterogeneous datasets. We present three inference algorithms for both estimating the factors and the additional parameters of the compound Poisson distribution. The methods are evaluated on two applications: modeling symbolic representations for polyphonic music and lyric prediction from audio features. Our conclusion is that the compound poisson based factorization models can be useful for sparse positive data.

Cite this Paper

BibTeX
@InProceedings{pmlr-v28-simsekli13, title = {Learning the beta-Divergence in Tweedie Compound Poisson Matrix Factorization Models}, author = {Simsekli, Umut and Taylan Cemgil, Ali and Kenan Yilmaz, Yusuf}, booktitle = {Proceedings of the 30th International Conference on Machine Learning}, pages = {1409--1417}, year = {2013}, editor = {Dasgupta, Sanjoy and McAllester, David}, volume = {28}, number = {3}, series = {Proceedings of Machine Learning Research}, address = {Atlanta, Georgia, USA}, month = {17--19 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v28/simsekli13.pdf}, url = {https://proceedings.mlr.press/v28/simsekli13.html}, abstract = {In this study, we derive algorithms for estimating mixed β-divergences. Such cost functions are useful for Nonnegative Matrix and Tensor Factorization models with a compound Poisson observation model. Compound Poisson is a particular Tweedie model, an important special case of exponential dispersion models characterized by the fact that the variance is proportional to a power function of the mean. There are several well known matrix and tensor factorization algorithms that minimize the β-divergence; these estimate the mean parameter. The probabilistic interpretation gives us more flexibility and robustness by providing us additional tunable parameters such as power and dispersion. Estimation of the power parameter is useful for choosing a suitable divergence and estimation of dispersion is useful for data driven regularization and weighting in collective/coupled factorization of heterogeneous datasets. We present three inference algorithms for both estimating the factors and the additional parameters of the compound Poisson distribution. The methods are evaluated on two applications: modeling symbolic representations for polyphonic music and lyric prediction from audio features. Our conclusion is that the compound poisson based factorization models can be useful for sparse positive data.} }
Endnote
%0 Conference Paper %T Learning the beta-Divergence in Tweedie Compound Poisson Matrix Factorization Models %A Umut Simsekli %A Ali Taylan Cemgil %A Yusuf Kenan Yilmaz %B Proceedings of the 30th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2013 %E Sanjoy Dasgupta %E David McAllester %F pmlr-v28-simsekli13 %I PMLR %P 1409--1417 %U https://proceedings.mlr.press/v28/simsekli13.html %V 28 %N 3 %X In this study, we derive algorithms for estimating mixed β-divergences. Such cost functions are useful for Nonnegative Matrix and Tensor Factorization models with a compound Poisson observation model. Compound Poisson is a particular Tweedie model, an important special case of exponential dispersion models characterized by the fact that the variance is proportional to a power function of the mean. There are several well known matrix and tensor factorization algorithms that minimize the β-divergence; these estimate the mean parameter. The probabilistic interpretation gives us more flexibility and robustness by providing us additional tunable parameters such as power and dispersion. Estimation of the power parameter is useful for choosing a suitable divergence and estimation of dispersion is useful for data driven regularization and weighting in collective/coupled factorization of heterogeneous datasets. We present three inference algorithms for both estimating the factors and the additional parameters of the compound Poisson distribution. The methods are evaluated on two applications: modeling symbolic representations for polyphonic music and lyric prediction from audio features. Our conclusion is that the compound poisson based factorization models can be useful for sparse positive data.
RIS
TY - CPAPER TI - Learning the beta-Divergence in Tweedie Compound Poisson Matrix Factorization Models AU - Umut Simsekli AU - Ali Taylan Cemgil AU - Yusuf Kenan Yilmaz BT - Proceedings of the 30th International Conference on Machine Learning DA - 2013/05/26 ED - Sanjoy Dasgupta ED - David McAllester ID - pmlr-v28-simsekli13 PB - PMLR DP - Proceedings of Machine Learning Research VL - 28 IS - 3 SP - 1409 EP - 1417 L1 - http://proceedings.mlr.press/v28/simsekli13.pdf UR - https://proceedings.mlr.press/v28/simsekli13.html AB - In this study, we derive algorithms for estimating mixed β-divergences. Such cost functions are useful for Nonnegative Matrix and Tensor Factorization models with a compound Poisson observation model. Compound Poisson is a particular Tweedie model, an important special case of exponential dispersion models characterized by the fact that the variance is proportional to a power function of the mean. There are several well known matrix and tensor factorization algorithms that minimize the β-divergence; these estimate the mean parameter. The probabilistic interpretation gives us more flexibility and robustness by providing us additional tunable parameters such as power and dispersion. Estimation of the power parameter is useful for choosing a suitable divergence and estimation of dispersion is useful for data driven regularization and weighting in collective/coupled factorization of heterogeneous datasets. We present three inference algorithms for both estimating the factors and the additional parameters of the compound Poisson distribution. The methods are evaluated on two applications: modeling symbolic representations for polyphonic music and lyric prediction from audio features. Our conclusion is that the compound poisson based factorization models can be useful for sparse positive data. ER -
APA
Simsekli, U., Taylan Cemgil, A. & Kenan Yilmaz, Y.. (2013). Learning the beta-Divergence in Tweedie Compound Poisson Matrix Factorization Models. Proceedings of the 30th International Conference on Machine Learning, in Proceedings of Machine Learning Research 28(3):1409-1417 Available from https://proceedings.mlr.press/v28/simsekli13.html.

Related Material