Performance of Kullback-Leibler Based Expert Opinion Pooling for Unlikely Events


Vladimı́ra Sečkárová ;
Proceedings of the NIPS 2016 Workshop on Imperfect Decision Makers, PMLR 58:41-50, 2017.


The aggregation of available information is of great importance in many branches of economics, social sciences. Often, we can only rely on experts’ opinions, i.e. probabilities assigned to possible events. To deal with opinions in probabilistic form, we focus on the Kullback-Leibler (KL) divergence based pools: linear, logarithmic and KL-pool. Since occurrence of events is subject to random influences of the real world, it is important to address events assigned lower probabilities (unlikely events). This is done by choosing pooling with a higher entropy than standard linear or logarithmic options, i.e. the KL-pool. We show how well the mentioned pools perform on real data using absolute error, KL-divergence and quadratic reward. In cases favoring events assigned higher probabilities, the KL-pool performs similarly to the linear pool and outperforms the logarithmic pool. When unlikely events occur, the KL-pool outperforms both pools, which makes it a reasonable way of pooling.

Related Material