Valid Inferential Models for Prediction in Supervised Learning Problems

Leonardo Cella, Ryan Martin
Proceedings of the Twelveth International Symposium on Imprecise Probability: Theories and Applications, PMLR 147:72-82, 2021.

Abstract

Prediction, where observed data is used to quantify uncertainty about a future observation, is a fundamental problem in statistics. Prediction sets with coverage probability guarantees are a common solution, but these do not provide probabilistic uncertainty quantification in the sense of assigning beliefs to relevant assertions about the future observable. Alternatively, we recommend the use of a probabilistic predictor, a fully-specified (imprecise) probability distribution for the to-be-predicted observation given the observed data. It is essential that the probabilistic predictor is reliable or valid in some sense, and here we offer a notion of validity and explore its implications. We also provide a general inferential model construction that yields a provably valid probabilistic predictor, with illustrations in regression and classification.

Cite this Paper

BibTeX
@InProceedings{pmlr-v147-cella21a, title = {Valid Inferential Models for Prediction in Supervised Learning Problems}, author = {Cella, Leonardo and Martin, Ryan}, booktitle = {Proceedings of the Twelveth International Symposium on Imprecise Probability: Theories and Applications}, pages = {72--82}, year = {2021}, editor = {Cano, Andrés and De Bock, Jasper and Miranda, Enrique and Moral, Serafı́n}, volume = {147}, series = {Proceedings of Machine Learning Research}, month = {06--09 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v147/cella21a/cella21a.pdf}, url = {https://proceedings.mlr.press/v147/cella21a.html}, abstract = {Prediction, where observed data is used to quantify uncertainty about a future observation, is a fundamental problem in statistics. Prediction sets with coverage probability guarantees are a common solution, but these do not provide probabilistic uncertainty quantification in the sense of assigning beliefs to relevant assertions about the future observable. Alternatively, we recommend the use of a probabilistic predictor, a fully-specified (imprecise) probability distribution for the to-be-predicted observation given the observed data. It is essential that the probabilistic predictor is reliable or valid in some sense, and here we offer a notion of validity and explore its implications. We also provide a general inferential model construction that yields a provably valid probabilistic predictor, with illustrations in regression and classification.} }
Endnote
%0 Conference Paper %T Valid Inferential Models for Prediction in Supervised Learning Problems %A Leonardo Cella %A Ryan Martin %B Proceedings of the Twelveth International Symposium on Imprecise Probability: Theories and Applications %C Proceedings of Machine Learning Research %D 2021 %E Andrés Cano %E Jasper De Bock %E Enrique Miranda %E Serafı́n Moral %F pmlr-v147-cella21a %I PMLR %P 72--82 %U https://proceedings.mlr.press/v147/cella21a.html %V 147 %X Prediction, where observed data is used to quantify uncertainty about a future observation, is a fundamental problem in statistics. Prediction sets with coverage probability guarantees are a common solution, but these do not provide probabilistic uncertainty quantification in the sense of assigning beliefs to relevant assertions about the future observable. Alternatively, we recommend the use of a probabilistic predictor, a fully-specified (imprecise) probability distribution for the to-be-predicted observation given the observed data. It is essential that the probabilistic predictor is reliable or valid in some sense, and here we offer a notion of validity and explore its implications. We also provide a general inferential model construction that yields a provably valid probabilistic predictor, with illustrations in regression and classification.
APA
Cella, L. & Martin, R.. (2021). Valid Inferential Models for Prediction in Supervised Learning Problems. Proceedings of the Twelveth International Symposium on Imprecise Probability: Theories and Applications, in Proceedings of Machine Learning Research 147:72-82 Available from https://proceedings.mlr.press/v147/cella21a.html.

Related Material