Learning Mixtures of Smooth Product Distributions: Identifiability and Algorithm

Nikos Kargas, Nicholas D. Sidiropoulos
Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics, PMLR 89:388-396, 2019.

Abstract

We study the problem of learning a mixture model of non-parametric product distributions. The problem of learning a mixture model is that of finding the component distributions along with the mixing weights using observed samples generated from the mixture. The problem is well-studied in the parametric setting, i.e., when the component distributions are members of a parametric family - such as Gaussian distributions. In this work, we focus on multivariate mixtures of non-parametric product distributions and propose a two-stage approach which recovers the component distributions of the mixture under a smoothness condition. Our approach builds upon the identifiability properties of the canonical polyadic (low-rank) decomposition of tensors, in tandem with Fourier and Shannon-Nyquist sampling staples from signal processing. We demonstrate the effectiveness of the approach on synthetic and real datasets.

Cite this Paper

BibTeX
@InProceedings{pmlr-v89-kargas19a, title = {Learning Mixtures of Smooth Product Distributions: Identifiability and Algorithm}, author = {Kargas, Nikos and Sidiropoulos, Nicholas D.}, booktitle = {Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics}, pages = {388--396}, year = {2019}, editor = {Chaudhuri, Kamalika and Sugiyama, Masashi}, volume = {89}, series = {Proceedings of Machine Learning Research}, month = {16--18 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v89/kargas19a/kargas19a.pdf}, url = {https://proceedings.mlr.press/v89/kargas19a.html}, abstract = {We study the problem of learning a mixture model of non-parametric product distributions. The problem of learning a mixture model is that of finding the component distributions along with the mixing weights using observed samples generated from the mixture. The problem is well-studied in the parametric setting, i.e., when the component distributions are members of a parametric family - such as Gaussian distributions. In this work, we focus on multivariate mixtures of non-parametric product distributions and propose a two-stage approach which recovers the component distributions of the mixture under a smoothness condition. Our approach builds upon the identifiability properties of the canonical polyadic (low-rank) decomposition of tensors, in tandem with Fourier and Shannon-Nyquist sampling staples from signal processing. We demonstrate the effectiveness of the approach on synthetic and real datasets.} }
Endnote
%0 Conference Paper %T Learning Mixtures of Smooth Product Distributions: Identifiability and Algorithm %A Nikos Kargas %A Nicholas D. Sidiropoulos %B Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2019 %E Kamalika Chaudhuri %E Masashi Sugiyama %F pmlr-v89-kargas19a %I PMLR %P 388--396 %U https://proceedings.mlr.press/v89/kargas19a.html %V 89 %X We study the problem of learning a mixture model of non-parametric product distributions. The problem of learning a mixture model is that of finding the component distributions along with the mixing weights using observed samples generated from the mixture. The problem is well-studied in the parametric setting, i.e., when the component distributions are members of a parametric family - such as Gaussian distributions. In this work, we focus on multivariate mixtures of non-parametric product distributions and propose a two-stage approach which recovers the component distributions of the mixture under a smoothness condition. Our approach builds upon the identifiability properties of the canonical polyadic (low-rank) decomposition of tensors, in tandem with Fourier and Shannon-Nyquist sampling staples from signal processing. We demonstrate the effectiveness of the approach on synthetic and real datasets.
APA
Kargas, N. & Sidiropoulos, N.D.. (2019). Learning Mixtures of Smooth Product Distributions: Identifiability and Algorithm. Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 89:388-396 Available from https://proceedings.mlr.press/v89/kargas19a.html.

Related Material