Decision Heuristics for Comparison:How Good Are They?

Marcus Buckmann, Özgür Şimşek
Proceedings of the NIPS 2016 Workshop on Imperfect Decision Makers, PMLR 58:1-11, 2017.

Abstract

Simple decision heuristics are cognitive models of human and animal decision making. They examine few pieces of information and combine the pieces in simple ways, for example, by considering them sequentially or giving them equal weight. They have been studied most extensively for the problem of \textitcomparison, where the objective is to identify which of a given number of alternatives has the highest value on a specified (unobserved) criterion. We present the most comprehensive empirical evaluation of decision heuristics to date on the comparison problem. In a diverse collection of 56 real-world data sets, we compared heuristics to powerful statistical learning methods, including support vector machines and random forests. Heuristics performed surprisingly well. On average, they were only a few percentage points behind the best-performing algorithm. In many data sets, they yielded the highest accuracy in all or parts of the learning curve. The first part of the supplement describes implementation details of the algorithms tested; the second part describes the 56 public data sets used in the empirical analysis.

Cite this Paper

BibTeX
@InProceedings{pmlr-v58-buckmann17a, title = {Decision Heuristics for Comparison:{H}ow Good Are They?}, author = {Buckmann, Marcus and Şimşek, Özgür}, booktitle = {Proceedings of the NIPS 2016 Workshop on Imperfect Decision Makers}, pages = {1--11}, year = {2017}, editor = {Guy, Tatiana V. and Kárný, Miroslav and Rios-Insua, David and Wolpert, David H.}, volume = {58}, series = {Proceedings of Machine Learning Research}, month = {09 Dec}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v58/buckmann17a/buckmann17a.pdf}, url = {https://proceedings.mlr.press/v58/buckmann17a.html}, abstract = {Simple decision heuristics are cognitive models of human and animal decision making. They examine few pieces of information and combine the pieces in simple ways, for example, by considering them sequentially or giving them equal weight. They have been studied most extensively for the problem of \textitcomparison, where the objective is to identify which of a given number of alternatives has the highest value on a specified (unobserved) criterion. We present the most comprehensive empirical evaluation of decision heuristics to date on the comparison problem. In a diverse collection of 56 real-world data sets, we compared heuristics to powerful statistical learning methods, including support vector machines and random forests. Heuristics performed surprisingly well. On average, they were only a few percentage points behind the best-performing algorithm. In many data sets, they yielded the highest accuracy in all or parts of the learning curve. The first part of the supplement describes implementation details of the algorithms tested; the second part describes the 56 public data sets used in the empirical analysis. } }
Endnote
%0 Conference Paper %T Decision Heuristics for Comparison:How Good Are They? %A Marcus Buckmann %A Özgür Şimşek %B Proceedings of the NIPS 2016 Workshop on Imperfect Decision Makers %C Proceedings of Machine Learning Research %D 2017 %E Tatiana V. Guy %E Miroslav Kárný %E David Rios-Insua %E David H. Wolpert %F pmlr-v58-buckmann17a %I PMLR %P 1--11 %U https://proceedings.mlr.press/v58/buckmann17a.html %V 58 %X Simple decision heuristics are cognitive models of human and animal decision making. They examine few pieces of information and combine the pieces in simple ways, for example, by considering them sequentially or giving them equal weight. They have been studied most extensively for the problem of \textitcomparison, where the objective is to identify which of a given number of alternatives has the highest value on a specified (unobserved) criterion. We present the most comprehensive empirical evaluation of decision heuristics to date on the comparison problem. In a diverse collection of 56 real-world data sets, we compared heuristics to powerful statistical learning methods, including support vector machines and random forests. Heuristics performed surprisingly well. On average, they were only a few percentage points behind the best-performing algorithm. In many data sets, they yielded the highest accuracy in all or parts of the learning curve. The first part of the supplement describes implementation details of the algorithms tested; the second part describes the 56 public data sets used in the empirical analysis.
APA
Buckmann, M. & Şimşek, Ö.. (2017). Decision Heuristics for Comparison:How Good Are They?. Proceedings of the NIPS 2016 Workshop on Imperfect Decision Makers, in Proceedings of Machine Learning Research 58:1-11 Available from https://proceedings.mlr.press/v58/buckmann17a.html.

Related Material