[edit]

# Simple Regression Models

*Proceedings of the NIPS 2016 Workshop on Imperfect Decision Makers*, PMLR 58:13-25, 2017.

#### Abstract

Developing theories of when and why simple predictive models
perform well is a key step in understanding decisions of cognitively
bounded humans and intelligent machines. We are interested in
how well simple models predict in regression. We list and review
existing simple regression models and define new ones. We identify
the lack of a large-scale empirical comparison of these models with
state-of-the-art regression models in a predictive regression context.
We report the results of such an empirical analysis on 60 real-world data
sets. Simple regression models such as equal-weights regression routinely
outperformed state-of-the-art regression models, especially on small training-set sizes.
There was no simple model that predicted well in all data sets, but in nearly
all data sets, there was at least one simple model that predicted well.
The supplementary material contains learning curves for individual data sets that have
not been presented in the main article. It also contains detailed descriptions and source
descriptions of all used data sets.