- title: 'NIPS Workshop on Imperfect Decision Makers 2016: Preface' abstract: 'The workshop aims, organising and programme committees, invited talks and panel discussions are introduced.' volume: 58 URL: https://proceedings.mlr.press/v58/guy17a.html PDF: http://proceedings.mlr.press/v58/guy17a/guy17a.pdf edit: https://github.com/mlresearch//v58/edit/gh-pages/_posts/2017-08-14-guy17a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the NIPS 2016 Workshop on Imperfect Decision Makers' publisher: 'PMLR' author: - given: Tatiana V. family: Guy editor: - given: Tatiana V. family: Guy - given: Miroslav family: Kárný - given: David family: Rios-Insua - given: David H. family: Wolpert page: i-iii id: guy17a issued: date-parts: - 2017 - 8 - 14 firstpage: i lastpage: iii published: 2017-08-14 00:00:00 +0000 - title: 'Decision Heuristics for Comparison:How Good Are They?' abstract: 'Simple decision heuristics are cognitive models of human and animal decision making. They examine few pieces of information and combine the pieces in simple ways, for example, by considering them sequentially or giving them equal weight. They have been studied most extensively for the problem of \textitcomparison, where the objective is to identify which of a given number of alternatives has the highest value on a specified (unobserved) criterion. We present the most comprehensive empirical evaluation of decision heuristics to date on the comparison problem. In a diverse collection of 56 real-world data sets, we compared heuristics to powerful statistical learning methods, including support vector machines and random forests. Heuristics performed surprisingly well. On average, they were only a few percentage points behind the best-performing algorithm. In many data sets, they yielded the highest accuracy in all or parts of the learning curve. The first part of the supplement describes implementation details of the algorithms tested; the second part describes the 56 public data sets used in the empirical analysis. ' volume: 58 URL: https://proceedings.mlr.press/v58/buckmann17a.html PDF: http://proceedings.mlr.press/v58/buckmann17a/buckmann17a.pdf edit: https://github.com/mlresearch//v58/edit/gh-pages/_posts/2017-08-14-buckmann17a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the NIPS 2016 Workshop on Imperfect Decision Makers' publisher: 'PMLR' author: - given: Marcus family: Buckmann - given: Özgür family: Şimşek editor: - given: Tatiana V. family: Guy - given: Miroslav family: Kárný - given: David family: Rios-Insua - given: David H. family: Wolpert page: 1-11 id: buckmann17a issued: date-parts: - 2017 - 8 - 14 firstpage: 1 lastpage: 11 published: 2017-08-14 00:00:00 +0000 - title: 'Simple Regression Models' abstract: 'Developing theories of when and why simple predictive models perform well is a key step in understanding decisions of cognitively bounded humans and intelligent machines. We are interested in how well simple models predict in regression. We list and review existing simple regression models and define new ones. We identify the lack of a large-scale empirical comparison of these models with state-of-the-art regression models in a predictive regression context. We report the results of such an empirical analysis on 60 real-world data sets. Simple regression models such as equal-weights regression routinely outperformed state-of-the-art regression models, especially on small training-set sizes. There was no simple model that predicted well in all data sets, but in nearly all data sets, there was at least one simple model that predicted well. The supplementary material contains learning curves for individual data sets that have not been presented in the main article. It also contains detailed descriptions and source descriptions of all used data sets. ' volume: 58 URL: https://proceedings.mlr.press/v58/lichtenberg17a.html PDF: http://proceedings.mlr.press/v58/lichtenberg17a/lichtenberg17a.pdf edit: https://github.com/mlresearch//v58/edit/gh-pages/_posts/2017-08-14-lichtenberg17a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the NIPS 2016 Workshop on Imperfect Decision Makers' publisher: 'PMLR' author: - given: Jan M. family: Lichtenberg - given: Özgür family: Şimşek editor: - given: Tatiana V. family: Guy - given: Miroslav family: Kárný - given: David family: Rios-Insua - given: David H. family: Wolpert page: 13-25 id: lichtenberg17a issued: date-parts: - 2017 - 8 - 14 firstpage: 13 lastpage: 25 published: 2017-08-14 00:00:00 +0000 - title: 'Implementable Prescriptive Decision Making' abstract: 'The need for inspecting (ir)rationality in decision making (DM) — the observed discrepancy between real and prescriptive DMs — stems from omnipresence of DM in individuals’ and society life. Active approaches try to diminish this discrepancy either by changing behaviour of participants (DM subjects) or modifying prescriptive theories as done in this text. It provides a core of unified merging methodology of probabilities serving for knowledge fusion and information sharing exploited in cooperative DM. Specifically, it unifies merging methodologies supporting a flat cooperation of interacting self-interested DM participants. They act without a facilitator and they are unwilling to spare a non-negligible deliberation effort on merging. They are supposed to solve their DM tasks via the fully probabilistic design (FPD) of decision strategies. This option is motivated by the fact that FPD is axiomatically justified and extends standard Bayesian DM. Merging is a supporting DM task and is also solved via FPD. The proposed merging formulation tries to be as general as possible without entering into technicalities of measure theory. The results generalise and unify earlier work and open a pathway to systematic solutions of specific, less general, problems.' volume: 58 URL: https://proceedings.mlr.press/v58/karny17a.html PDF: http://proceedings.mlr.press/v58/karny17a/karny17a.pdf edit: https://github.com/mlresearch//v58/edit/gh-pages/_posts/2017-08-14-karny17a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the NIPS 2016 Workshop on Imperfect Decision Makers' publisher: 'PMLR' author: - given: Miroslav family: Kárný editor: - given: Tatiana V. family: Guy - given: Miroslav family: Kárný - given: David family: Rios-Insua - given: David H. family: Wolpert page: 19-30 id: karny17a issued: date-parts: - 2017 - 8 - 14 firstpage: 19 lastpage: 30 published: 2017-08-14 00:00:00 +0000 - title: 'Forecasting Spot Oil Price Using Google Probabilities' abstract: 'In this paper DMA (Dynamic Averaging Model) is expanded by adding certain probabilities based on Google Trends. Such a method is applied to forecasting spot oil prices (WTI). In particular it is checked whether a dynamic model including stock prices in developed markets, stock prices in China, stock prices volatility, exchange rates, global economic activity, interest rates, production, consumption, import and level of inventories as independent variables might be improved by including a certain measure of Google searches. Monthly data between 2004 and 2015 were analysed. It was found that such a modification leads to slightly better forecast. However, the weight ascribed to Google searches should be quite small. Except that it was found that even unmodified DMA produced better forecast than that based on futures contracts or naive forecast.' volume: 58 URL: https://proceedings.mlr.press/v58/drachal17a.html PDF: http://proceedings.mlr.press/v58/drachal17a/drachal17a.pdf edit: https://github.com/mlresearch//v58/edit/gh-pages/_posts/2017-08-14-drachal17a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the NIPS 2016 Workshop on Imperfect Decision Makers' publisher: 'PMLR' author: - given: Krzysztof family: Drachal editor: - given: Tatiana V. family: Guy - given: Miroslav family: Kárný - given: David family: Rios-Insua - given: David H. family: Wolpert page: 31-40 id: drachal17a issued: date-parts: - 2017 - 8 - 14 firstpage: 31 lastpage: 40 published: 2017-08-14 00:00:00 +0000 - title: 'Performance of Kullback-Leibler Based Expert Opinion Pooling for Unlikely Events' abstract: 'The aggregation of available information is of great importance in many branches of economics, social sciences. Often, we can only rely on experts’ opinions, i.e. probabilities assigned to possible events. To deal with opinions in probabilistic form, we focus on the Kullback-Leibler (KL) divergence based pools: linear, logarithmic and KL-pool. Since occurrence of events is subject to random influences of the real world, it is important to address events assigned lower probabilities (unlikely events). This is done by choosing pooling with a higher entropy than standard linear or logarithmic options, i.e. the KL-pool. We show how well the mentioned pools perform on real data using absolute error, KL-divergence and quadratic reward. In cases favoring events assigned higher probabilities, the KL-pool performs similarly to the linear pool and outperforms the logarithmic pool. When unlikely events occur, the KL-pool outperforms both pools, which makes it a reasonable way of pooling.' volume: 58 URL: https://proceedings.mlr.press/v58/seckarova17a.html PDF: http://proceedings.mlr.press/v58/seckarova17a/seckarova17a.pdf edit: https://github.com/mlresearch//v58/edit/gh-pages/_posts/2017-08-14-seckarova17a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the NIPS 2016 Workshop on Imperfect Decision Makers' publisher: 'PMLR' author: - given: Vladimı́ra family: Sečkárová editor: - given: Tatiana V. family: Guy - given: Miroslav family: Kárný - given: David family: Rios-Insua - given: David H. family: Wolpert page: 41-50 id: seckarova17a issued: date-parts: - 2017 - 8 - 14 firstpage: 41 lastpage: 50 published: 2017-08-14 00:00:00 +0000 - title: 'Experimental Performance of Deliberation-Aware Responder in Multi-Proposer Ultimatum Game' abstract: 'The ultimatum game serves for studying various aspects of decision making (DM). Recently, its multi-proposer version has been modified to study the influence of deliberation costs. An optimising policy of the responder, switching between several proposers at non-negligible deliberation costs, was designed and successfully tested in a simulated environment. The policy design was done within the framework of Markov Decision Processes with rewards also allowing to model the responder’s feeling for fairness. It relies on simple Markov models of proposers, which are recursively learnt in a Bayesian way during the game course. This paper verifies, whether the gained theoretically plausible policy, suits to real-life DM. It describes experiments in which this policy was applied against human proposers. The results – with eleven groups of three independently acting proposers – confirm the soundness of this policy. It increases the responder’s economic profit due to switching between proposers, in spite of the deliberation costs and the used approximate modelling of proposers. Methodologically, it opens the possibility to learn systematically willingness of humans to spent their deliberation resources on specific DM tasks.' volume: 58 URL: https://proceedings.mlr.press/v58/guy17b.html PDF: http://proceedings.mlr.press/v58/guy17b/guy17b.pdf edit: https://github.com/mlresearch//v58/edit/gh-pages/_posts/2017-08-14-guy17b.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the NIPS 2016 Workshop on Imperfect Decision Makers' publisher: 'PMLR' author: - given: Tatiana V. family: Guy - given: Marko family: Ruman - given: František family: Hůla - given: Miroslav family: Kárný editor: - given: Tatiana V. family: Guy - given: Miroslav family: Kárný - given: David family: Rios-Insua - given: David H. family: Wolpert page: 51-60 id: guy17b issued: date-parts: - 2017 - 8 - 14 firstpage: 51 lastpage: 60 published: 2017-08-14 00:00:00 +0000 - title: 'Optimal Number of Choices in Rating Contexts' abstract: 'In many settings people must give numerical scores to entities from a small discrete set. For instance, rating physical attractiveness from 1–5 on dating sites, or papers from 1–10 for conference reviewing. We study the problem of understanding when using a different number of options is optimal. For concreteness we assume the true underlying scores are integers from 1–100. We consider the case when scores are uniform random and Gaussian. While in theory for this setting it would be optimal to use all 100 options, in practice this is prohibitive, and it is preferable to utilize a smaller number of options due to humans’ cognitive limitations. Our results suggest that using a smaller number of options than is typical could be optimal in certain situations. This would have many potential applications, as settings requiring entities to be ranked by humans are ubiquitous.' volume: 58 URL: https://proceedings.mlr.press/v58/ganzfried17a.html PDF: http://proceedings.mlr.press/v58/ganzfried17a/ganzfried17a.pdf edit: https://github.com/mlresearch//v58/edit/gh-pages/_posts/2017-08-14-ganzfried17a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the NIPS 2016 Workshop on Imperfect Decision Makers' publisher: 'PMLR' author: - given: Sam family: Ganzfried editor: - given: Tatiana V. family: Guy - given: Miroslav family: Kárný - given: David family: Rios-Insua - given: David H. family: Wolpert page: 61-74 id: ganzfried17a issued: date-parts: - 2017 - 8 - 14 firstpage: 61 lastpage: 74 published: 2017-08-14 00:00:00 +0000 - title: 'On Learning Decision Heuristics' abstract: 'Decision heuristics are simple models of human and animal decision making that use few pieces of information and combine the pieces in simple ways, for example, by giving them equal weight or by considering them sequentially. We examine how decision heuristics can be learned—and modified—as additional training examples become available. In particular, we examine how additional training examples change the variance in parameter estimates of the heuristic. Our analysis suggests new decision heuristics, including a family of heuristics that generalizes two well-known families: lexicographic heuristics and tallying. We evaluate the empirical performance of these heuristics in a large, diverse collection of data sets. The supplementary material provides details on the random forest implementation and describes the 56 public data sets used in the empirical analysis. ' volume: 58 URL: https://proceedings.mlr.press/v58/simsek17a.html PDF: http://proceedings.mlr.press/v58/simsek17a/simsek17a.pdf edit: https://github.com/mlresearch//v58/edit/gh-pages/_posts/2017-08-14-simsek17a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the NIPS 2016 Workshop on Imperfect Decision Makers' publisher: 'PMLR' author: - given: Özgür family: Şimşek - given: Marcus family: Buckmann editor: - given: Tatiana V. family: Guy - given: Miroslav family: Kárný - given: David family: Rios-Insua - given: David H. family: Wolpert page: 75-85 id: simsek17a issued: date-parts: - 2017 - 8 - 14 firstpage: 75 lastpage: 85 published: 2017-08-14 00:00:00 +0000 - title: 'Quantum Rational Preferences and Desirability' abstract: 'We develop a theory of quantum rational decision making in the tradition of Anscombe and Aumann’s axiomatisation of preferences on horse lotteries. It is essentially the Bayesian decision theory generalised to the space of Hermitian matrices. Among other things, this leads us to give a representation theorem showing that quantum complete rational preferences are obtained by means of expected utility considerations.' volume: 58 URL: https://proceedings.mlr.press/v58/benavoli17a.html PDF: http://proceedings.mlr.press/v58/benavoli17a/benavoli17a.pdf edit: https://github.com/mlresearch//v58/edit/gh-pages/_posts/2017-08-14-benavoli17a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the NIPS 2016 Workshop on Imperfect Decision Makers' publisher: 'PMLR' author: - given: Alessio family: Benavoli - given: Alessandro family: Facchini - given: Marco family: Zaffalon editor: - given: Tatiana V. family: Guy - given: Miroslav family: Kárný - given: David family: Rios-Insua - given: David H. family: Wolpert page: 87-96 id: benavoli17a issued: date-parts: - 2017 - 8 - 14 firstpage: 87 lastpage: 96 published: 2017-08-14 00:00:00 +0000 - title: 'Rational Beliefs Real Agents Can Have – A Logical Point of View' abstract: 'The purpose of this note is to outline a framework for uncertain reasoning which drops unrealistic assumptions about the agents’ inferential capabilities. To do so, we envisage a pivotal role for the recent research programme of \emphdepth-bounded Boolean logics. We suggest that this can be fruitfully extended to the representation of rational belief under uncertainty. By doing this we lay the foundations for a prescriptive account of rational belief, namely one that \emphrealistic agents, as opposed to idealised ones, can feasibly act upon.' volume: 58 URL: https://proceedings.mlr.press/v58/d-agostino17a.html PDF: http://proceedings.mlr.press/v58/d-agostino17a/d-agostino17a.pdf edit: https://github.com/mlresearch//v58/edit/gh-pages/_posts/2017-08-14-d-agostino17a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the NIPS 2016 Workshop on Imperfect Decision Makers' publisher: 'PMLR' author: - given: Marcello family: D’Agostino - given: Tommaso family: Flaminio - given: Hykel family: Hosni editor: - given: Tatiana V. family: Guy - given: Miroslav family: Kárný - given: David family: Rios-Insua - given: David H. family: Wolpert page: 97-109 id: d-agostino17a issued: date-parts: - 2017 - 8 - 14 firstpage: 97 lastpage: 109 published: 2017-08-14 00:00:00 +0000 - title: 'Hindsight Bias Impedes Learning' abstract: 'We propose a model that addresses an open question in the cognitive science literature: How can we rigorously model the cognitive bias known as hindsight bias such that we fully account for critical experimental results? Though hindsight bias has been studied extensively, prior work has failed to produce a consensus theoretical model sufficiently general to account for several key experimental results, or to fully demonstrate how hindsight impedes our ability to learn the truth in a repeated decision or social network setting. We present a model in which agents aim to learn the quality of their signals through repeated interactions with their environment. Our results indicate that agents who are subject to hindsight bias will always believe themselves to be high-type “expert” regardless of whether they are actually high- or low-type.' volume: 58 URL: https://proceedings.mlr.press/v58/mahdavi17a.html PDF: http://proceedings.mlr.press/v58/mahdavi17a/mahdavi17a.pdf edit: https://github.com/mlresearch//v58/edit/gh-pages/_posts/2017-08-14-mahdavi17a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the NIPS 2016 Workshop on Imperfect Decision Makers' publisher: 'PMLR' author: - given: Shaudi family: Mahdavi - given: M. Amin family: Rahimian editor: - given: Tatiana V. family: Guy - given: Miroslav family: Kárný - given: David family: Rios-Insua - given: David H. family: Wolpert page: 111-127 id: mahdavi17a issued: date-parts: - 2017 - 8 - 14 firstpage: 111 lastpage: 127 published: 2017-08-14 00:00:00 +0000