Performance of Kullback-Leibler Based Expert Opinion Pooling for Unlikely Events

Vladimı́ra Sečkárová

Performance of Kullback-Leibler Based Expert Opinion Pooling for Unlikely Events

Vladimı́ra Sečkárová ;

Proceedings of the NIPS 2016 Workshop on Imperfect Decision Makers, PMLR 58:41-50, 2017.

Abstract

The aggregation of available information is of great importance in many branches of economics, social sciences. Often, we can only rely on experts’ opinions, i.e. probabilities assigned to possible events. To deal with opinions in probabilistic form, we focus on the Kullback-Leibler (KL) divergence based pools: linear, logarithmic and KL-pool. Since occurrence of events is subject to random influences of the real world, it is important to address events assigned lower probabilities (unlikely events). This is done by choosing pooling with a higher entropy than standard linear or logarithmic options, i.e. the KL-pool. We show how well the mentioned pools perform on real data using absolute error, KL-divergence and quadratic reward. In cases favoring events assigned higher probabilities, the KL-pool performs similarly to the linear pool and outperforms the logarithmic pool. When unlikely events occur, the KL-pool outperforms both pools, which makes it a reasonable way of pooling.

Related Material

Download PDF

@InProceedings{pmlr-v58-seckarova17a, title = {Performance of {K}ullback-{L}eibler Based Expert Opinion Pooling for Unlikely Events}, author = {Vladimı́ra Sečkárová}, booktitle = {Proceedings of the NIPS 2016 Workshop on Imperfect Decision Makers}, pages = {41--50}, year = {2017}, editor = {Tatiana V. Guy and Miroslav Kárný and David Rios-Insua and David H. Wolpert}, volume = {58}, series = {Proceedings of Machine Learning Research}, address = {Centre de Convencions Internacional de Barcelona, Barcelona, Spain}, month = {09 Dec}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v58/seckarova17a/seckarova17a.pdf}, url = {http://proceedings.mlr.press/v58/seckarova17a.html}, abstract = {The aggregation of available information is of great importance in many branches of economics, social sciences. Often, we can only rely on experts’ opinions, i.e. probabilities assigned to possible events. To deal with opinions in probabilistic form, we focus on the Kullback-Leibler (KL) divergence based pools: linear, logarithmic and KL-pool. Since occurrence of events is subject to random influences of the real world, it is important to address events assigned lower probabilities (unlikely events). This is done by choosing pooling with a higher entropy than standard linear or logarithmic options, i.e. the KL-pool. We show how well the mentioned pools perform on real data using absolute error, KL-divergence and quadratic reward. In cases favoring events assigned higher probabilities, the KL-pool performs similarly to the linear pool and outperforms the logarithmic pool. When unlikely events occur, the KL-pool outperforms both pools, which makes it a reasonable way of pooling.} }

%0 Conference Paper %T Performance of Kullback-Leibler Based Expert Opinion Pooling for Unlikely Events %A Vladimı́ra Sečkárová %B Proceedings of the NIPS 2016 Workshop on Imperfect Decision Makers %C Proceedings of Machine Learning Research %D 2017 %E Tatiana V. Guy %E Miroslav Kárný %E David Rios-Insua %E David H. Wolpert %F pmlr-v58-seckarova17a %I PMLR %J Proceedings of Machine Learning Research %P 41--50 %U http://proceedings.mlr.press %V 58 %W PMLR %X The aggregation of available information is of great importance in many branches of economics, social sciences. Often, we can only rely on experts’ opinions, i.e. probabilities assigned to possible events. To deal with opinions in probabilistic form, we focus on the Kullback-Leibler (KL) divergence based pools: linear, logarithmic and KL-pool. Since occurrence of events is subject to random influences of the real world, it is important to address events assigned lower probabilities (unlikely events). This is done by choosing pooling with a higher entropy than standard linear or logarithmic options, i.e. the KL-pool. We show how well the mentioned pools perform on real data using absolute error, KL-divergence and quadratic reward. In cases favoring events assigned higher probabilities, the KL-pool performs similarly to the linear pool and outperforms the logarithmic pool. When unlikely events occur, the KL-pool outperforms both pools, which makes it a reasonable way of pooling.