Spectral M-estimation with Applications to Hidden Markov Models

Dustin Tran, Minjae Kim, Finale Doshi-Velez
Proceedings of the 19th International Conference on Artificial Intelligence and Statistics, PMLR 51:1421-1430, 2016.

Abstract

Method of moment estimators exhibit appealing statistical properties, such as asymptotic unbiasedness, for nonconvex problems. However, they typically require a large number of samples and are extremely sensitive to model misspecification. In this paper, we apply the framework of M-estimation to develop both a generalized method of moments procedure and a principled method for regularization. Our proposed M-estimator obtains optimal sample efficiency rates (in the class of moment-based estimators) and the same well-known rates on prediction accuracy as other spectral estimators. It also makes it straightforward to incorporate regularization into the sample moment conditions. We demonstrate empirically the gains in sample efficiency from our approach on hidden Markov models.

Cite this Paper

BibTeX
@InProceedings{pmlr-v51-tran16, title = {Spectral M-estimation with Applications to Hidden Markov Models}, author = {Tran, Dustin and Kim, Minjae and Doshi-Velez, Finale}, booktitle = {Proceedings of the 19th International Conference on Artificial Intelligence and Statistics}, pages = {1421--1430}, year = {2016}, editor = {Gretton, Arthur and Robert, Christian C.}, volume = {51}, series = {Proceedings of Machine Learning Research}, address = {Cadiz, Spain}, month = {09--11 May}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v51/tran16.pdf}, url = {https://proceedings.mlr.press/v51/tran16.html}, abstract = {Method of moment estimators exhibit appealing statistical properties, such as asymptotic unbiasedness, for nonconvex problems. However, they typically require a large number of samples and are extremely sensitive to model misspecification. In this paper, we apply the framework of M-estimation to develop both a generalized method of moments procedure and a principled method for regularization. Our proposed M-estimator obtains optimal sample efficiency rates (in the class of moment-based estimators) and the same well-known rates on prediction accuracy as other spectral estimators. It also makes it straightforward to incorporate regularization into the sample moment conditions. We demonstrate empirically the gains in sample efficiency from our approach on hidden Markov models.} }
Endnote
%0 Conference Paper %T Spectral M-estimation with Applications to Hidden Markov Models %A Dustin Tran %A Minjae Kim %A Finale Doshi-Velez %B Proceedings of the 19th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2016 %E Arthur Gretton %E Christian C. Robert %F pmlr-v51-tran16 %I PMLR %P 1421--1430 %U https://proceedings.mlr.press/v51/tran16.html %V 51 %X Method of moment estimators exhibit appealing statistical properties, such as asymptotic unbiasedness, for nonconvex problems. However, they typically require a large number of samples and are extremely sensitive to model misspecification. In this paper, we apply the framework of M-estimation to develop both a generalized method of moments procedure and a principled method for regularization. Our proposed M-estimator obtains optimal sample efficiency rates (in the class of moment-based estimators) and the same well-known rates on prediction accuracy as other spectral estimators. It also makes it straightforward to incorporate regularization into the sample moment conditions. We demonstrate empirically the gains in sample efficiency from our approach on hidden Markov models.
RIS
TY - CPAPER TI - Spectral M-estimation with Applications to Hidden Markov Models AU - Dustin Tran AU - Minjae Kim AU - Finale Doshi-Velez BT - Proceedings of the 19th International Conference on Artificial Intelligence and Statistics DA - 2016/05/02 ED - Arthur Gretton ED - Christian C. Robert ID - pmlr-v51-tran16 PB - PMLR DP - Proceedings of Machine Learning Research VL - 51 SP - 1421 EP - 1430 L1 - http://proceedings.mlr.press/v51/tran16.pdf UR - https://proceedings.mlr.press/v51/tran16.html AB - Method of moment estimators exhibit appealing statistical properties, such as asymptotic unbiasedness, for nonconvex problems. However, they typically require a large number of samples and are extremely sensitive to model misspecification. In this paper, we apply the framework of M-estimation to develop both a generalized method of moments procedure and a principled method for regularization. Our proposed M-estimator obtains optimal sample efficiency rates (in the class of moment-based estimators) and the same well-known rates on prediction accuracy as other spectral estimators. It also makes it straightforward to incorporate regularization into the sample moment conditions. We demonstrate empirically the gains in sample efficiency from our approach on hidden Markov models. ER -
APA
Tran, D., Kim, M. & Doshi-Velez, F.. (2016). Spectral M-estimation with Applications to Hidden Markov Models. Proceedings of the 19th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 51:1421-1430 Available from https://proceedings.mlr.press/v51/tran16.html.

Related Material