Learning Binary Latent Variable Models: A Tensor Eigenpair Approach

Ariel Jaffe, Roi Weiss, Boaz Nadler, Shai Carmi, Yuval Kluger
Proceedings of the 35th International Conference on Machine Learning, PMLR 80:2196-2205, 2018.

Abstract

Latent variable models with hidden binary units appear in various applications. Learning such models, in particular in the presence of noise, is a challenging computational problem. In this paper we propose a novel spectral approach to this problem, based on the eigenvectors of both the second order moment matrix and third order moment tensor of the observed data. We prove that under mild non-degeneracy conditions, our method consistently estimates the model parameters at the optimal parametric rate. Our tensor-based method generalizes previous orthogonal tensor decomposition approaches, where the hidden units were assumed to be either statistically independent or mutually exclusive. We illustrate the consistency of our method on simulated data and demonstrate its usefulness in learning a common model for population mixtures in genetics.

Cite this Paper

BibTeX
@InProceedings{pmlr-v80-jaffe18a, title = {Learning Binary Latent Variable Models: A Tensor Eigenpair Approach}, author = {Jaffe, Ariel and Weiss, Roi and Nadler, Boaz and Carmi, Shai and Kluger, Yuval}, booktitle = {Proceedings of the 35th International Conference on Machine Learning}, pages = {2196--2205}, year = {2018}, editor = {Dy, Jennifer and Krause, Andreas}, volume = {80}, series = {Proceedings of Machine Learning Research}, month = {10--15 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v80/jaffe18a/jaffe18a.pdf}, url = {https://proceedings.mlr.press/v80/jaffe18a.html}, abstract = {Latent variable models with hidden binary units appear in various applications. Learning such models, in particular in the presence of noise, is a challenging computational problem. In this paper we propose a novel spectral approach to this problem, based on the eigenvectors of both the second order moment matrix and third order moment tensor of the observed data. We prove that under mild non-degeneracy conditions, our method consistently estimates the model parameters at the optimal parametric rate. Our tensor-based method generalizes previous orthogonal tensor decomposition approaches, where the hidden units were assumed to be either statistically independent or mutually exclusive. We illustrate the consistency of our method on simulated data and demonstrate its usefulness in learning a common model for population mixtures in genetics.} }
Endnote
%0 Conference Paper %T Learning Binary Latent Variable Models: A Tensor Eigenpair Approach %A Ariel Jaffe %A Roi Weiss %A Boaz Nadler %A Shai Carmi %A Yuval Kluger %B Proceedings of the 35th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2018 %E Jennifer Dy %E Andreas Krause %F pmlr-v80-jaffe18a %I PMLR %P 2196--2205 %U https://proceedings.mlr.press/v80/jaffe18a.html %V 80 %X Latent variable models with hidden binary units appear in various applications. Learning such models, in particular in the presence of noise, is a challenging computational problem. In this paper we propose a novel spectral approach to this problem, based on the eigenvectors of both the second order moment matrix and third order moment tensor of the observed data. We prove that under mild non-degeneracy conditions, our method consistently estimates the model parameters at the optimal parametric rate. Our tensor-based method generalizes previous orthogonal tensor decomposition approaches, where the hidden units were assumed to be either statistically independent or mutually exclusive. We illustrate the consistency of our method on simulated data and demonstrate its usefulness in learning a common model for population mixtures in genetics.
APA
Jaffe, A., Weiss, R., Nadler, B., Carmi, S. & Kluger, Y.. (2018). Learning Binary Latent Variable Models: A Tensor Eigenpair Approach. Proceedings of the 35th International Conference on Machine Learning, in Proceedings of Machine Learning Research 80:2196-2205 Available from https://proceedings.mlr.press/v80/jaffe18a.html.

Related Material