Online Learning with Continuous Ranked Probability Score

Vladimir V. V’yugin, Vladimir G. Trunov
Proceedings of the Eighth Symposium on Conformal and Probabilistic Prediction and Applications, PMLR 105:163-177, 2019.

Abstract

Probabilistic forecasts in the form of probability distributions over future events have become popular in several fields of statistical science. The dissimilarity between a probability forecast and an outcome is measured by a loss function (scoring rule). Popular example of scoring rule for continuous outcomes is the continuous ranked probability score (CRPS). We consider the case where several competing methods produce online predictions in the form of probability distribution functions. In this paper, the problem of combining probabilistic forecasts is considered in the prediction with expert advice framework. We show that CRPS is a mixable loss function and then the time-independent upper bound for the regret of the Vovk’s Aggregating Algorithm using CRPS as a loss function can be obtained. We present the results of numerical experiments illustrating the proposed methods.

Cite this Paper

BibTeX
@InProceedings{pmlr-v105-v-yugin19a, title = {Online Learning with Continuous Ranked Probability Score}, author = {V'yugin, Vladimir V. and Trunov, Vladimir G.}, booktitle = {Proceedings of the Eighth Symposium on Conformal and Probabilistic Prediction and Applications}, pages = {163--177}, year = {2019}, editor = {Gammerman, Alex and Vovk, Vladimir and Luo, Zhiyuan and Smirnov, Evgueni}, volume = {105}, series = {Proceedings of Machine Learning Research}, month = {09--11 Sep}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v105/v-yugin19a/v-yugin19a.pdf}, url = {https://proceedings.mlr.press/v105/v-yugin19a.html}, abstract = {Probabilistic forecasts in the form of probability distributions over future events have become popular in several fields of statistical science. The dissimilarity between a probability forecast and an outcome is measured by a loss function (scoring rule). Popular example of scoring rule for continuous outcomes is the continuous ranked probability score (CRPS). We consider the case where several competing methods produce online predictions in the form of probability distribution functions. In this paper, the problem of combining probabilistic forecasts is considered in the prediction with expert advice framework. We show that CRPS is a mixable loss function and then the time-independent upper bound for the regret of the Vovk’s Aggregating Algorithm using CRPS as a loss function can be obtained. We present the results of numerical experiments illustrating the proposed methods.} }
Endnote
%0 Conference Paper %T Online Learning with Continuous Ranked Probability Score %A Vladimir V. V’yugin %A Vladimir G. Trunov %B Proceedings of the Eighth Symposium on Conformal and Probabilistic Prediction and Applications %C Proceedings of Machine Learning Research %D 2019 %E Alex Gammerman %E Vladimir Vovk %E Zhiyuan Luo %E Evgueni Smirnov %F pmlr-v105-v-yugin19a %I PMLR %P 163--177 %U https://proceedings.mlr.press/v105/v-yugin19a.html %V 105 %X Probabilistic forecasts in the form of probability distributions over future events have become popular in several fields of statistical science. The dissimilarity between a probability forecast and an outcome is measured by a loss function (scoring rule). Popular example of scoring rule for continuous outcomes is the continuous ranked probability score (CRPS). We consider the case where several competing methods produce online predictions in the form of probability distribution functions. In this paper, the problem of combining probabilistic forecasts is considered in the prediction with expert advice framework. We show that CRPS is a mixable loss function and then the time-independent upper bound for the regret of the Vovk’s Aggregating Algorithm using CRPS as a loss function can be obtained. We present the results of numerical experiments illustrating the proposed methods.
APA
V’yugin, V.V. & Trunov, V.G.. (2019). Online Learning with Continuous Ranked Probability Score. Proceedings of the Eighth Symposium on Conformal and Probabilistic Prediction and Applications, in Proceedings of Machine Learning Research 105:163-177 Available from https://proceedings.mlr.press/v105/v-yugin19a.html.

Related Material