Fast Robust Logistic Regression for Large Sparse Datasets with Binary Outputs

Paul Komarek, Andrew W. Moore
Proceedings of the Ninth International Workshop on Artificial Intelligence and Statistics, PMLR R4:163-170, 2003.

Abstract

Although popular and extremely well established in mainstream statistical data analysis, logistic regression is strangely absent in the field of data mining. There are two possible explanations of this phenomenon. First, there might be an assumption that any tool which can only produce linear classification boundaries is likely to be trumped by more modern nonlinear tools. Second, there is a legitimate fear that logistic regression cannot practically scale up to the massive dataset sizes to which modern data mining tools are applied. This paper consists of an empirical examination of the first assumption, and surveys, implements and compares techniques by which logistic regression can be scaled to data with millions of attributes and records. Our results, on a large life sciences dataset, indicate that logistic regression can perform surprisingly well, both statistically and computationally, when compared with an array of more recent classification algorithms.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR4-komarek03a, title = {Fast Robust Logistic Regression for Large Sparse Datasets with Binary Outputs}, author = {Komarek, Paul and Moore, Andrew W.}, booktitle = {Proceedings of the Ninth International Workshop on Artificial Intelligence and Statistics}, pages = {163--170}, year = {2003}, editor = {Bishop, Christopher M. and Frey, Brendan J.}, volume = {R4}, series = {Proceedings of Machine Learning Research}, month = {03--06 Jan}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/r4/komarek03a/komarek03a.pdf}, url = {https://proceedings.mlr.press/r4/komarek03a.html}, abstract = {Although popular and extremely well established in mainstream statistical data analysis, logistic regression is strangely absent in the field of data mining. There are two possible explanations of this phenomenon. First, there might be an assumption that any tool which can only produce linear classification boundaries is likely to be trumped by more modern nonlinear tools. Second, there is a legitimate fear that logistic regression cannot practically scale up to the massive dataset sizes to which modern data mining tools are applied. This paper consists of an empirical examination of the first assumption, and surveys, implements and compares techniques by which logistic regression can be scaled to data with millions of attributes and records. Our results, on a large life sciences dataset, indicate that logistic regression can perform surprisingly well, both statistically and computationally, when compared with an array of more recent classification algorithms.}, note = {Reissued by PMLR on 01 April 2021.} }
Endnote
%0 Conference Paper %T Fast Robust Logistic Regression for Large Sparse Datasets with Binary Outputs %A Paul Komarek %A Andrew W. Moore %B Proceedings of the Ninth International Workshop on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2003 %E Christopher M. Bishop %E Brendan J. Frey %F pmlr-vR4-komarek03a %I PMLR %P 163--170 %U https://proceedings.mlr.press/r4/komarek03a.html %V R4 %X Although popular and extremely well established in mainstream statistical data analysis, logistic regression is strangely absent in the field of data mining. There are two possible explanations of this phenomenon. First, there might be an assumption that any tool which can only produce linear classification boundaries is likely to be trumped by more modern nonlinear tools. Second, there is a legitimate fear that logistic regression cannot practically scale up to the massive dataset sizes to which modern data mining tools are applied. This paper consists of an empirical examination of the first assumption, and surveys, implements and compares techniques by which logistic regression can be scaled to data with millions of attributes and records. Our results, on a large life sciences dataset, indicate that logistic regression can perform surprisingly well, both statistically and computationally, when compared with an array of more recent classification algorithms. %Z Reissued by PMLR on 01 April 2021.
APA
Komarek, P. & Moore, A.W.. (2003). Fast Robust Logistic Regression for Large Sparse Datasets with Binary Outputs. Proceedings of the Ninth International Workshop on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research R4:163-170 Available from https://proceedings.mlr.press/r4/komarek03a.html. Reissued by PMLR on 01 April 2021.

Related Material