Competitive Online Regression under Continuous Ranked Probability Score

Raisa Dzhamtyrova, Yuri Kalnishkan
Proceedings of the Eighth Symposium on Conformal and Probabilistic Prediction and Applications, PMLR 105:178-195, 2019.

Abstract

We consider the framework of competitive prediction when one provides guarantees compared to other predictive models that are called experts. We propose the algorithm that combines point predictions of an infinite pool of linear experts and outputs probability forecasts in the form of cumulative distribution functions. We evaluate the quality of probabilistic prediction by the continuous ranked probability score (CRPS), which is a widely used proper scoring rule. We provide a strategy that allows us to “track the best expert” and derive the theoretical bound on the discounted loss of the strategy. Experimental results on synthetic data and solar power data show that the theoretical bounds of our algorithm are not violated. Also the algorithm performs close to and sometimes outperforms the retrospectively best quantile regression.

Cite this Paper

BibTeX
@InProceedings{pmlr-v105-dzhamtyrova19a, title = {Competitive Online Regression under Continuous Ranked Probability Score}, author = {Dzhamtyrova, Raisa and Kalnishkan, Yuri}, booktitle = {Proceedings of the Eighth Symposium on Conformal and Probabilistic Prediction and Applications}, pages = {178--195}, year = {2019}, editor = {Gammerman, Alex and Vovk, Vladimir and Luo, Zhiyuan and Smirnov, Evgueni}, volume = {105}, series = {Proceedings of Machine Learning Research}, month = {09--11 Sep}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v105/dzhamtyrova19a/dzhamtyrova19a.pdf}, url = {https://proceedings.mlr.press/v105/dzhamtyrova19a.html}, abstract = {We consider the framework of competitive prediction when one provides guarantees compared to other predictive models that are called experts. We propose the algorithm that combines point predictions of an infinite pool of linear experts and outputs probability forecasts in the form of cumulative distribution functions. We evaluate the quality of probabilistic prediction by the continuous ranked probability score (CRPS), which is a widely used proper scoring rule. We provide a strategy that allows us to “track the best expert” and derive the theoretical bound on the discounted loss of the strategy. Experimental results on synthetic data and solar power data show that the theoretical bounds of our algorithm are not violated. Also the algorithm performs close to and sometimes outperforms the retrospectively best quantile regression.} }
Endnote
%0 Conference Paper %T Competitive Online Regression under Continuous Ranked Probability Score %A Raisa Dzhamtyrova %A Yuri Kalnishkan %B Proceedings of the Eighth Symposium on Conformal and Probabilistic Prediction and Applications %C Proceedings of Machine Learning Research %D 2019 %E Alex Gammerman %E Vladimir Vovk %E Zhiyuan Luo %E Evgueni Smirnov %F pmlr-v105-dzhamtyrova19a %I PMLR %P 178--195 %U https://proceedings.mlr.press/v105/dzhamtyrova19a.html %V 105 %X We consider the framework of competitive prediction when one provides guarantees compared to other predictive models that are called experts. We propose the algorithm that combines point predictions of an infinite pool of linear experts and outputs probability forecasts in the form of cumulative distribution functions. We evaluate the quality of probabilistic prediction by the continuous ranked probability score (CRPS), which is a widely used proper scoring rule. We provide a strategy that allows us to “track the best expert” and derive the theoretical bound on the discounted loss of the strategy. Experimental results on synthetic data and solar power data show that the theoretical bounds of our algorithm are not violated. Also the algorithm performs close to and sometimes outperforms the retrospectively best quantile regression.
APA
Dzhamtyrova, R. & Kalnishkan, Y.. (2019). Competitive Online Regression under Continuous Ranked Probability Score. Proceedings of the Eighth Symposium on Conformal and Probabilistic Prediction and Applications, in Proceedings of Machine Learning Research 105:178-195 Available from https://proceedings.mlr.press/v105/dzhamtyrova19a.html.

Related Material