Simple Regression Models

Jan M. Lichtenberg, Özgür Şimşek
Proceedings of the NIPS 2016 Workshop on Imperfect Decision Makers, PMLR 58:13-25, 2017.

Abstract

Developing theories of when and why simple predictive models perform well is a key step in understanding decisions of cognitively bounded humans and intelligent machines. We are interested in how well simple models predict in regression. We list and review existing simple regression models and define new ones. We identify the lack of a large-scale empirical comparison of these models with state-of-the-art regression models in a predictive regression context. We report the results of such an empirical analysis on 60 real-world data sets. Simple regression models such as equal-weights regression routinely outperformed state-of-the-art regression models, especially on small training-set sizes. There was no simple model that predicted well in all data sets, but in nearly all data sets, there was at least one simple model that predicted well. The supplementary material contains learning curves for individual data sets that have not been presented in the main article. It also contains detailed descriptions and source descriptions of all used data sets.

Cite this Paper

BibTeX
@InProceedings{pmlr-v58-lichtenberg17a, title = {Simple Regression Models}, author = {Lichtenberg, Jan M. and Şimşek, Özgür}, booktitle = {Proceedings of the NIPS 2016 Workshop on Imperfect Decision Makers}, pages = {13--25}, year = {2017}, editor = {Guy, Tatiana V. and Kárný, Miroslav and Rios-Insua, David and Wolpert, David H.}, volume = {58}, series = {Proceedings of Machine Learning Research}, month = {09 Dec}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v58/lichtenberg17a/lichtenberg17a.pdf}, url = {https://proceedings.mlr.press/v58/lichtenberg17a.html}, abstract = {Developing theories of when and why simple predictive models perform well is a key step in understanding decisions of cognitively bounded humans and intelligent machines. We are interested in how well simple models predict in regression. We list and review existing simple regression models and define new ones. We identify the lack of a large-scale empirical comparison of these models with state-of-the-art regression models in a predictive regression context. We report the results of such an empirical analysis on 60 real-world data sets. Simple regression models such as equal-weights regression routinely outperformed state-of-the-art regression models, especially on small training-set sizes. There was no simple model that predicted well in all data sets, but in nearly all data sets, there was at least one simple model that predicted well. The supplementary material contains learning curves for individual data sets that have not been presented in the main article. It also contains detailed descriptions and source descriptions of all used data sets. } }
Endnote
%0 Conference Paper %T Simple Regression Models %A Jan M. Lichtenberg %A Özgür Şimşek %B Proceedings of the NIPS 2016 Workshop on Imperfect Decision Makers %C Proceedings of Machine Learning Research %D 2017 %E Tatiana V. Guy %E Miroslav Kárný %E David Rios-Insua %E David H. Wolpert %F pmlr-v58-lichtenberg17a %I PMLR %P 13--25 %U https://proceedings.mlr.press/v58/lichtenberg17a.html %V 58 %X Developing theories of when and why simple predictive models perform well is a key step in understanding decisions of cognitively bounded humans and intelligent machines. We are interested in how well simple models predict in regression. We list and review existing simple regression models and define new ones. We identify the lack of a large-scale empirical comparison of these models with state-of-the-art regression models in a predictive regression context. We report the results of such an empirical analysis on 60 real-world data sets. Simple regression models such as equal-weights regression routinely outperformed state-of-the-art regression models, especially on small training-set sizes. There was no simple model that predicted well in all data sets, but in nearly all data sets, there was at least one simple model that predicted well. The supplementary material contains learning curves for individual data sets that have not been presented in the main article. It also contains detailed descriptions and source descriptions of all used data sets.
APA
Lichtenberg, J.M. & Şimşek, Ö.. (2017). Simple Regression Models. Proceedings of the NIPS 2016 Workshop on Imperfect Decision Makers, in Proceedings of Machine Learning Research 58:13-25 Available from https://proceedings.mlr.press/v58/lichtenberg17a.html.

Related Material