Estimation of Non-Normalized Mixture Models

Takeru Matsuda, Aapo Hyvärinen
Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics, PMLR 89:2555-2563, 2019.

Abstract

We develop a general method for estimating a finite mixture of non-normalized models. A non-normalized model is defined to be a parametric distribution with an intractable normalization constant. Existing methods for estimating non-normalized models without computing the normalization constant are not applicable to mixture models because they contain more than one intractable normalization constant. The proposed method is derived by extending noise contrastive estimation (NCE), which estimates non-normalized models by discriminating between the observed data and some artificially generated noise. In particular, the proposed method provides a probabilistically principled clustering method that is able to utilize a deep representation. Applications to clustering of natural images and neuroimaging data give promising results.

Cite this Paper

BibTeX
@InProceedings{pmlr-v89-matsuda19a, title = {Estimation of Non-Normalized Mixture Models}, author = {Matsuda, Takeru and Hyv\"{a}rinen, Aapo}, booktitle = {Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics}, pages = {2555--2563}, year = {2019}, editor = {Chaudhuri, Kamalika and Sugiyama, Masashi}, volume = {89}, series = {Proceedings of Machine Learning Research}, month = {16--18 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v89/matsuda19a/matsuda19a.pdf}, url = {https://proceedings.mlr.press/v89/matsuda19a.html}, abstract = {We develop a general method for estimating a finite mixture of non-normalized models. A non-normalized model is defined to be a parametric distribution with an intractable normalization constant. Existing methods for estimating non-normalized models without computing the normalization constant are not applicable to mixture models because they contain more than one intractable normalization constant. The proposed method is derived by extending noise contrastive estimation (NCE), which estimates non-normalized models by discriminating between the observed data and some artificially generated noise. In particular, the proposed method provides a probabilistically principled clustering method that is able to utilize a deep representation. Applications to clustering of natural images and neuroimaging data give promising results.} }
Endnote
%0 Conference Paper %T Estimation of Non-Normalized Mixture Models %A Takeru Matsuda %A Aapo Hyvärinen %B Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2019 %E Kamalika Chaudhuri %E Masashi Sugiyama %F pmlr-v89-matsuda19a %I PMLR %P 2555--2563 %U https://proceedings.mlr.press/v89/matsuda19a.html %V 89 %X We develop a general method for estimating a finite mixture of non-normalized models. A non-normalized model is defined to be a parametric distribution with an intractable normalization constant. Existing methods for estimating non-normalized models without computing the normalization constant are not applicable to mixture models because they contain more than one intractable normalization constant. The proposed method is derived by extending noise contrastive estimation (NCE), which estimates non-normalized models by discriminating between the observed data and some artificially generated noise. In particular, the proposed method provides a probabilistically principled clustering method that is able to utilize a deep representation. Applications to clustering of natural images and neuroimaging data give promising results.
APA
Matsuda, T. & Hyvärinen, A.. (2019). Estimation of Non-Normalized Mixture Models. Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 89:2555-2563 Available from https://proceedings.mlr.press/v89/matsuda19a.html.

Related Material