Inductive Venn-Abers predictive distribution

Ilia Nouretdinov, Denis Volkhonskiy, Pitt Lim, Paolo Toccaceli, Alexander Gammerman
Proceedings of the Seventh Workshop on Conformal and Probabilistic Prediction and Applications, PMLR 91:15-36, 2018.

Abstract

Venn predictors are a distribution-free probabilistic prediction framework that transforms the output of a scoring classifier into a (multi-)probabilistic prediction that has calibration guarantees, with the only requirement of an i.i.d. assumption for calibration and test data. In this paper, we extend the framework from classification (where probabilities are predicted for a discrete number of labels) to regression (where labels form a continuum). We show how Venn Predictors can be applied on top of any regression method to obtain calibrated predictive distributions, without requiring assumptions beyond i.i.d. of calibration and test sets. This is contrasted with methods such as Bayesian Linear Regression, for which the calibration guarantee instead relies on specific probabilistic assumptions on the distribution of the data. The adaptation of Venn Machine to regression required a theoretical analysis of the transductive and inductive forms of the predictor. We identify potential consistency problems and provide solutions for them. Finally, to illustrate their advantages, we apply regression Venn Predictors to the medical problem of predicting the survival time after Percutaneous Coronary Intervention, a potentially risky procedure that improves blood flow to a patient’s heart. The predictive distributions obtained with this method allow a variety of interpretations that include probability of survival time exceeding a chosen threshold or the shortest survival time guaranteed with a given probability.

Cite this Paper

BibTeX
@InProceedings{pmlr-v91-nouretdinov18a, title = {Inductive {V}enn-{A}bers predictive distribution}, author = {Nouretdinov, Ilia and Volkhonskiy, Denis and Lim, Pitt and Toccaceli, Paolo and Gammerman, Alexander}, booktitle = {Proceedings of the Seventh Workshop on Conformal and Probabilistic Prediction and Applications}, pages = {15--36}, year = {2018}, editor = {Gammerman, Alex and Vovk, Vladimir and Luo, Zhiyuan and Smirnov, Evgueni and Peeters, Ralf}, volume = {91}, series = {Proceedings of Machine Learning Research}, month = {11--13 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v91/nouretdinov18a/nouretdinov18a.pdf}, url = {https://proceedings.mlr.press/v91/nouretdinov18a.html}, abstract = {Venn predictors are a distribution-free probabilistic prediction framework that transforms the output of a scoring classifier into a (multi-)probabilistic prediction that has calibration guarantees, with the only requirement of an i.i.d. assumption for calibration and test data. In this paper, we extend the framework from classification (where probabilities are predicted for a discrete number of labels) to regression (where labels form a continuum). We show how Venn Predictors can be applied on top of any regression method to obtain calibrated predictive distributions, without requiring assumptions beyond i.i.d. of calibration and test sets. This is contrasted with methods such as Bayesian Linear Regression, for which the calibration guarantee instead relies on specific probabilistic assumptions on the distribution of the data. The adaptation of Venn Machine to regression required a theoretical analysis of the transductive and inductive forms of the predictor. We identify potential consistency problems and provide solutions for them. Finally, to illustrate their advantages, we apply regression Venn Predictors to the medical problem of predicting the survival time after Percutaneous Coronary Intervention, a potentially risky procedure that improves blood flow to a patient’s heart. The predictive distributions obtained with this method allow a variety of interpretations that include probability of survival time exceeding a chosen threshold or the shortest survival time guaranteed with a given probability.} }
Endnote
%0 Conference Paper %T Inductive Venn-Abers predictive distribution %A Ilia Nouretdinov %A Denis Volkhonskiy %A Pitt Lim %A Paolo Toccaceli %A Alexander Gammerman %B Proceedings of the Seventh Workshop on Conformal and Probabilistic Prediction and Applications %C Proceedings of Machine Learning Research %D 2018 %E Alex Gammerman %E Vladimir Vovk %E Zhiyuan Luo %E Evgueni Smirnov %E Ralf Peeters %F pmlr-v91-nouretdinov18a %I PMLR %P 15--36 %U https://proceedings.mlr.press/v91/nouretdinov18a.html %V 91 %X Venn predictors are a distribution-free probabilistic prediction framework that transforms the output of a scoring classifier into a (multi-)probabilistic prediction that has calibration guarantees, with the only requirement of an i.i.d. assumption for calibration and test data. In this paper, we extend the framework from classification (where probabilities are predicted for a discrete number of labels) to regression (where labels form a continuum). We show how Venn Predictors can be applied on top of any regression method to obtain calibrated predictive distributions, without requiring assumptions beyond i.i.d. of calibration and test sets. This is contrasted with methods such as Bayesian Linear Regression, for which the calibration guarantee instead relies on specific probabilistic assumptions on the distribution of the data. The adaptation of Venn Machine to regression required a theoretical analysis of the transductive and inductive forms of the predictor. We identify potential consistency problems and provide solutions for them. Finally, to illustrate their advantages, we apply regression Venn Predictors to the medical problem of predicting the survival time after Percutaneous Coronary Intervention, a potentially risky procedure that improves blood flow to a patient’s heart. The predictive distributions obtained with this method allow a variety of interpretations that include probability of survival time exceeding a chosen threshold or the shortest survival time guaranteed with a given probability.
APA
Nouretdinov, I., Volkhonskiy, D., Lim, P., Toccaceli, P. & Gammerman, A.. (2018). Inductive Venn-Abers predictive distribution. Proceedings of the Seventh Workshop on Conformal and Probabilistic Prediction and Applications, in Proceedings of Machine Learning Research 91:15-36 Available from https://proceedings.mlr.press/v91/nouretdinov18a.html.

Related Material