Learning Latent Variable Models via Method of Moments and Exterior Point Optimization

Amirreza Shaban Georgia Institute of Technolog, Mehrdad Farajtabar Georgia Institute of Technology, Bo Xie Georgia Institute of Technology, Le Song Georgia Tech, Byron Boots Georgia Institute of Technology
Proceedings of the 31st Conference on Uncertainty in Artificial Intelligence, PMLR R13:940-949, 2015.

Abstract

Probabilistic latent-variable models are a fundamental tool in statistics and machine learning. Despite their widespread use, identifying the parameters of basic latent variable models continues to be an extremely challenging problem. Traditional maximum likelihood-based learning algorithms find valid parameters, but suffer from high computational cost, slow convergence, and local optima. In contrast, recently developed method of moments-based algorithms are computationally efficient and provide strong statistical guarantees, but are not guaranteed to find valid parameters. In this work, we introduce a two-stage learning algorithm for latent variable models. We first use method of moments to find a solution that is close to the optimal solution but not necessarily in the valid set of model parameters. We then incrementally refine the solution via exterior point optimization until a local optima that is arbitrarily near the valid set of parameters is found. We perform several experiments on synthetic and real-world data and show that our approach is more accurate then previous work, especially when training data is limited.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR13-technolog15a, title = {Learning Latent Variable Models via Method of Moments and Exterior Point Optimization}, author = {Technolog, Amirreza Shaban Georgia Institute of and Technology, Mehrdad Farajtabar Georgia Institute of and Technology, Bo Xie Georgia Institute of and Tech, Le Song Georgia and Technology, Byron Boots Georgia Institute of}, booktitle = {Proceedings of the 31st Conference on Uncertainty in Artificial Intelligence}, pages = {940--949}, year = {2015}, editor = {Meila, Marina and Heskes, Tom}, volume = {R13}, series = {Proceedings of Machine Learning Research}, month = {12--16 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r13/main/assets/technolog15a/technolog15a.pdf}, url = {https://proceedings.mlr.press/r13/technolog15a.html}, abstract = {Probabilistic latent-variable models are a fundamental tool in statistics and machine learning. Despite their widespread use, identifying the parameters of basic latent variable models continues to be an extremely challenging problem. Traditional maximum likelihood-based learning algorithms find valid parameters, but suffer from high computational cost, slow convergence, and local optima. In contrast, recently developed method of moments-based algorithms are computationally efficient and provide strong statistical guarantees, but are not guaranteed to find valid parameters. In this work, we introduce a two-stage learning algorithm for latent variable models. We first use method of moments to find a solution that is close to the optimal solution but not necessarily in the valid set of model parameters. We then incrementally refine the solution via exterior point optimization until a local optima that is arbitrarily near the valid set of parameters is found. We perform several experiments on synthetic and real-world data and show that our approach is more accurate then previous work, especially when training data is limited.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T Learning Latent Variable Models via Method of Moments and Exterior Point Optimization %A Amirreza Shaban Georgia Institute of Technolog %A Mehrdad Farajtabar Georgia Institute of Technology %A Bo Xie Georgia Institute of Technology %A Le Song Georgia Tech %A Byron Boots Georgia Institute of Technology %B Proceedings of the 31st Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2015 %E Marina Meila %E Tom Heskes %F pmlr-vR13-technolog15a %I PMLR %P 940--949 %U https://proceedings.mlr.press/r13/technolog15a.html %V R13 %X Probabilistic latent-variable models are a fundamental tool in statistics and machine learning. Despite their widespread use, identifying the parameters of basic latent variable models continues to be an extremely challenging problem. Traditional maximum likelihood-based learning algorithms find valid parameters, but suffer from high computational cost, slow convergence, and local optima. In contrast, recently developed method of moments-based algorithms are computationally efficient and provide strong statistical guarantees, but are not guaranteed to find valid parameters. In this work, we introduce a two-stage learning algorithm for latent variable models. We first use method of moments to find a solution that is close to the optimal solution but not necessarily in the valid set of model parameters. We then incrementally refine the solution via exterior point optimization until a local optima that is arbitrarily near the valid set of parameters is found. We perform several experiments on synthetic and real-world data and show that our approach is more accurate then previous work, especially when training data is limited. %Z Reissued by PMLR on 04 October 2026.
APA
Technolog, A.S.G.I.o., Technology, M.F.G.I.o., Technology, B.X.G.I.o., Tech, L.S.G. & Technology, B.B.G.I.o.. (2015). Learning Latent Variable Models via Method of Moments and Exterior Point Optimization. Proceedings of the 31st Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R13:940-949 Available from https://proceedings.mlr.press/r13/technolog15a.html. Reissued by PMLR on 04 October 2026.

Related Material