Spectral Experts for Estimating Mixtures of Linear Regressions

Arun Tejasvi Chaganty, Percy Liang
Proceedings of the 30th International Conference on Machine Learning, PMLR 28(3):1040-1048, 2013.

Abstract

Discriminative latent-variable models are typically learned using EM or gradient-based optimization, which suffer from local optima. In this paper, we develop a new computationally efficient and provably consistent estimator for the mixture of linear regressions, a simple instance of discriminative latent-variable models. Our approach relies on a low-rank linear regression to recover a symmetric tensor, which can be factorized into the parameters using the tensor power method. We prove rates of convergence for our estimator and provide an empirical evaluation illustrating its strengths relative to local optimization (EM).

Cite this Paper

BibTeX
@InProceedings{pmlr-v28-tejasvichaganty13, title = {Spectral Experts for Estimating Mixtures of Linear Regressions}, author = {Tejasvi Chaganty, Arun and Liang, Percy}, booktitle = {Proceedings of the 30th International Conference on Machine Learning}, pages = {1040--1048}, year = {2013}, editor = {Dasgupta, Sanjoy and McAllester, David}, volume = {28}, number = {3}, series = {Proceedings of Machine Learning Research}, address = {Atlanta, Georgia, USA}, month = {17--19 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v28/tejasvichaganty13.pdf}, url = {https://proceedings.mlr.press/v28/tejasvichaganty13.html}, abstract = {Discriminative latent-variable models are typically learned using EM or gradient-based optimization, which suffer from local optima. In this paper, we develop a new computationally efficient and provably consistent estimator for the mixture of linear regressions, a simple instance of discriminative latent-variable models. Our approach relies on a low-rank linear regression to recover a symmetric tensor, which can be factorized into the parameters using the tensor power method. We prove rates of convergence for our estimator and provide an empirical evaluation illustrating its strengths relative to local optimization (EM). } }
Endnote
%0 Conference Paper %T Spectral Experts for Estimating Mixtures of Linear Regressions %A Arun Tejasvi Chaganty %A Percy Liang %B Proceedings of the 30th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2013 %E Sanjoy Dasgupta %E David McAllester %F pmlr-v28-tejasvichaganty13 %I PMLR %P 1040--1048 %U https://proceedings.mlr.press/v28/tejasvichaganty13.html %V 28 %N 3 %X Discriminative latent-variable models are typically learned using EM or gradient-based optimization, which suffer from local optima. In this paper, we develop a new computationally efficient and provably consistent estimator for the mixture of linear regressions, a simple instance of discriminative latent-variable models. Our approach relies on a low-rank linear regression to recover a symmetric tensor, which can be factorized into the parameters using the tensor power method. We prove rates of convergence for our estimator and provide an empirical evaluation illustrating its strengths relative to local optimization (EM).
RIS
TY - CPAPER TI - Spectral Experts for Estimating Mixtures of Linear Regressions AU - Arun Tejasvi Chaganty AU - Percy Liang BT - Proceedings of the 30th International Conference on Machine Learning DA - 2013/05/26 ED - Sanjoy Dasgupta ED - David McAllester ID - pmlr-v28-tejasvichaganty13 PB - PMLR DP - Proceedings of Machine Learning Research VL - 28 IS - 3 SP - 1040 EP - 1048 L1 - http://proceedings.mlr.press/v28/tejasvichaganty13.pdf UR - https://proceedings.mlr.press/v28/tejasvichaganty13.html AB - Discriminative latent-variable models are typically learned using EM or gradient-based optimization, which suffer from local optima. In this paper, we develop a new computationally efficient and provably consistent estimator for the mixture of linear regressions, a simple instance of discriminative latent-variable models. Our approach relies on a low-rank linear regression to recover a symmetric tensor, which can be factorized into the parameters using the tensor power method. We prove rates of convergence for our estimator and provide an empirical evaluation illustrating its strengths relative to local optimization (EM). ER -
APA
Tejasvi Chaganty, A. & Liang, P.. (2013). Spectral Experts for Estimating Mixtures of Linear Regressions. Proceedings of the 30th International Conference on Machine Learning, in Proceedings of Machine Learning Research 28(3):1040-1048 Available from https://proceedings.mlr.press/v28/tejasvichaganty13.html.

Related Material