Performance of Kullback-Leibler Based Expert Opinion Pooling for Unlikely Events

Vladimı́ra Sečkárová
Proceedings of the NIPS 2016 Workshop on Imperfect Decision Makers, PMLR 58:41-50, 2017.

Abstract

The aggregation of available information is of great importance in many branches of economics, social sciences. Often, we can only rely on experts’ opinions, i.e. probabilities assigned to possible events. To deal with opinions in probabilistic form, we focus on the Kullback-Leibler (KL) divergence based pools: linear, logarithmic and KL-pool. Since occurrence of events is subject to random influences of the real world, it is important to address events assigned lower probabilities (unlikely events). This is done by choosing pooling with a higher entropy than standard linear or logarithmic options, i.e. the KL-pool. We show how well the mentioned pools perform on real data using absolute error, KL-divergence and quadratic reward. In cases favoring events assigned higher probabilities, the KL-pool performs similarly to the linear pool and outperforms the logarithmic pool. When unlikely events occur, the KL-pool outperforms both pools, which makes it a reasonable way of pooling.

Cite this Paper

BibTeX
@InProceedings{pmlr-v58-seckarova17a, title = {Performance of {K}ullback-{L}eibler Based Expert Opinion Pooling for Unlikely Events}, author = {Sečkárová, Vladimı́ra}, booktitle = {Proceedings of the NIPS 2016 Workshop on Imperfect Decision Makers}, pages = {41--50}, year = {2017}, editor = {Guy, Tatiana V. and Kárný, Miroslav and Rios-Insua, David and Wolpert, David H.}, volume = {58}, series = {Proceedings of Machine Learning Research}, month = {09 Dec}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v58/seckarova17a/seckarova17a.pdf}, url = {https://proceedings.mlr.press/v58/seckarova17a.html}, abstract = {The aggregation of available information is of great importance in many branches of economics, social sciences. Often, we can only rely on experts’ opinions, i.e. probabilities assigned to possible events. To deal with opinions in probabilistic form, we focus on the Kullback-Leibler (KL) divergence based pools: linear, logarithmic and KL-pool. Since occurrence of events is subject to random influences of the real world, it is important to address events assigned lower probabilities (unlikely events). This is done by choosing pooling with a higher entropy than standard linear or logarithmic options, i.e. the KL-pool. We show how well the mentioned pools perform on real data using absolute error, KL-divergence and quadratic reward. In cases favoring events assigned higher probabilities, the KL-pool performs similarly to the linear pool and outperforms the logarithmic pool. When unlikely events occur, the KL-pool outperforms both pools, which makes it a reasonable way of pooling.} }
Endnote
%0 Conference Paper %T Performance of Kullback-Leibler Based Expert Opinion Pooling for Unlikely Events %A Vladimı́ra Sečkárová %B Proceedings of the NIPS 2016 Workshop on Imperfect Decision Makers %C Proceedings of Machine Learning Research %D 2017 %E Tatiana V. Guy %E Miroslav Kárný %E David Rios-Insua %E David H. Wolpert %F pmlr-v58-seckarova17a %I PMLR %P 41--50 %U https://proceedings.mlr.press/v58/seckarova17a.html %V 58 %X The aggregation of available information is of great importance in many branches of economics, social sciences. Often, we can only rely on experts’ opinions, i.e. probabilities assigned to possible events. To deal with opinions in probabilistic form, we focus on the Kullback-Leibler (KL) divergence based pools: linear, logarithmic and KL-pool. Since occurrence of events is subject to random influences of the real world, it is important to address events assigned lower probabilities (unlikely events). This is done by choosing pooling with a higher entropy than standard linear or logarithmic options, i.e. the KL-pool. We show how well the mentioned pools perform on real data using absolute error, KL-divergence and quadratic reward. In cases favoring events assigned higher probabilities, the KL-pool performs similarly to the linear pool and outperforms the logarithmic pool. When unlikely events occur, the KL-pool outperforms both pools, which makes it a reasonable way of pooling.
APA
Sečkárová, V.. (2017). Performance of Kullback-Leibler Based Expert Opinion Pooling for Unlikely Events. Proceedings of the NIPS 2016 Workshop on Imperfect Decision Makers, in Proceedings of Machine Learning Research 58:41-50 Available from https://proceedings.mlr.press/v58/seckarova17a.html.

Related Material