Fast, Exact Model Selection and Permutation Testing for l2-Regularized Logistic Regression

Bryan Conroy, Paul Sajda
Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, PMLR 22:246-254, 2012.

Abstract

Regularized logistic regression is a standard classification method used in statistics and machine learning. Unlike regularized least squares problems such as ridge regression, the parameter estimates cannot be computed in closed-form and instead must be estimated using an iterative technique. This paper addresses the computational problem of regularized logistic regression that is commonly encountered in model selection and classifier statistical significance testing, in which a large number of related logistic regression problems must be solved for. Our proposed approach solves the problems simultaneously through an iterative technique, which also garners computational efficiencies by leveraging the redundancies across the related problems. We demonstrate analytically that our method provides a substantial complexity reduction, which is further validated by our results on real-world datasets.

Cite this Paper

BibTeX
@InProceedings{pmlr-v22-conroy12, title = {Fast, Exact Model Selection and Permutation Testing for l2-Regularized Logistic Regression}, author = {Conroy, Bryan and Sajda, Paul}, booktitle = {Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics}, pages = {246--254}, year = {2012}, editor = {Lawrence, Neil D. and Girolami, Mark}, volume = {22}, series = {Proceedings of Machine Learning Research}, address = {La Palma, Canary Islands}, month = {21--23 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v22/conroy12/conroy12.pdf}, url = {https://proceedings.mlr.press/v22/conroy12.html}, abstract = {Regularized logistic regression is a standard classification method used in statistics and machine learning. Unlike regularized least squares problems such as ridge regression, the parameter estimates cannot be computed in closed-form and instead must be estimated using an iterative technique. This paper addresses the computational problem of regularized logistic regression that is commonly encountered in model selection and classifier statistical significance testing, in which a large number of related logistic regression problems must be solved for. Our proposed approach solves the problems simultaneously through an iterative technique, which also garners computational efficiencies by leveraging the redundancies across the related problems. We demonstrate analytically that our method provides a substantial complexity reduction, which is further validated by our results on real-world datasets.} }
Endnote
%0 Conference Paper %T Fast, Exact Model Selection and Permutation Testing for l2-Regularized Logistic Regression %A Bryan Conroy %A Paul Sajda %B Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2012 %E Neil D. Lawrence %E Mark Girolami %F pmlr-v22-conroy12 %I PMLR %P 246--254 %U https://proceedings.mlr.press/v22/conroy12.html %V 22 %X Regularized logistic regression is a standard classification method used in statistics and machine learning. Unlike regularized least squares problems such as ridge regression, the parameter estimates cannot be computed in closed-form and instead must be estimated using an iterative technique. This paper addresses the computational problem of regularized logistic regression that is commonly encountered in model selection and classifier statistical significance testing, in which a large number of related logistic regression problems must be solved for. Our proposed approach solves the problems simultaneously through an iterative technique, which also garners computational efficiencies by leveraging the redundancies across the related problems. We demonstrate analytically that our method provides a substantial complexity reduction, which is further validated by our results on real-world datasets.
RIS
TY - CPAPER TI - Fast, Exact Model Selection and Permutation Testing for l2-Regularized Logistic Regression AU - Bryan Conroy AU - Paul Sajda BT - Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics DA - 2012/03/21 ED - Neil D. Lawrence ED - Mark Girolami ID - pmlr-v22-conroy12 PB - PMLR DP - Proceedings of Machine Learning Research VL - 22 SP - 246 EP - 254 L1 - http://proceedings.mlr.press/v22/conroy12/conroy12.pdf UR - https://proceedings.mlr.press/v22/conroy12.html AB - Regularized logistic regression is a standard classification method used in statistics and machine learning. Unlike regularized least squares problems such as ridge regression, the parameter estimates cannot be computed in closed-form and instead must be estimated using an iterative technique. This paper addresses the computational problem of regularized logistic regression that is commonly encountered in model selection and classifier statistical significance testing, in which a large number of related logistic regression problems must be solved for. Our proposed approach solves the problems simultaneously through an iterative technique, which also garners computational efficiencies by leveraging the redundancies across the related problems. We demonstrate analytically that our method provides a substantial complexity reduction, which is further validated by our results on real-world datasets. ER -
APA
Conroy, B. & Sajda, P.. (2012). Fast, Exact Model Selection and Permutation Testing for l2-Regularized Logistic Regression. Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 22:246-254 Available from https://proceedings.mlr.press/v22/conroy12.html.

Related Material