How do the performance of a Conformal Predictor and its underlying algorithm relate?

Giovanni Cherubin
Proceedings of the Twelfth Symposium on Conformal and Probabilistic Prediction with Applications, PMLR 204:546-548, 2023.

Abstract

Conformal Prediction (CP) offers a shift on the traditional supervised classification paradigm. Whereas in supervised learning one generally aims to optimize the error of a classifier at predicting the label correctly (prediction error), in CP one aims to optimize the size of a prediction set (efficiency), where this set is guaranteed to contain the true label with probability $\geq 1-\varepsilon$, for a user-defined $\varepsilon \in[0,1]$. CP works as a wrapper around a traditional learning model; yet, it is unclear how the prediction error of the underlying model affects the efficiency of the CP. In this note, we study a simple class of CPs whose efficiency is proportional to the prediction error of the underlying model.

Cite this Paper

BibTeX
@InProceedings{pmlr-v204-cherubin23a, title = {How do the performance of a Conformal Predictor and its underlying algorithm relate?}, author = {Cherubin, Giovanni}, booktitle = {Proceedings of the Twelfth Symposium on Conformal and Probabilistic Prediction with Applications}, pages = {546--548}, year = {2023}, editor = {Papadopoulos, Harris and Nguyen, Khuong An and Boström, Henrik and Carlsson, Lars}, volume = {204}, series = {Proceedings of Machine Learning Research}, month = {13--15 Sep}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v204/cherubin23a/cherubin23a.pdf}, url = {https://proceedings.mlr.press/v204/cherubin23a.html}, abstract = {Conformal Prediction (CP) offers a shift on the traditional supervised classification paradigm. Whereas in supervised learning one generally aims to optimize the error of a classifier at predicting the label correctly (prediction error), in CP one aims to optimize the size of a prediction set (efficiency), where this set is guaranteed to contain the true label with probability $\geq 1-\varepsilon$, for a user-defined $\varepsilon \in[0,1]$. CP works as a wrapper around a traditional learning model; yet, it is unclear how the prediction error of the underlying model affects the efficiency of the CP. In this note, we study a simple class of CPs whose efficiency is proportional to the prediction error of the underlying model.} }
Endnote
%0 Conference Paper %T How do the performance of a Conformal Predictor and its underlying algorithm relate? %A Giovanni Cherubin %B Proceedings of the Twelfth Symposium on Conformal and Probabilistic Prediction with Applications %C Proceedings of Machine Learning Research %D 2023 %E Harris Papadopoulos %E Khuong An Nguyen %E Henrik Boström %E Lars Carlsson %F pmlr-v204-cherubin23a %I PMLR %P 546--548 %U https://proceedings.mlr.press/v204/cherubin23a.html %V 204 %X Conformal Prediction (CP) offers a shift on the traditional supervised classification paradigm. Whereas in supervised learning one generally aims to optimize the error of a classifier at predicting the label correctly (prediction error), in CP one aims to optimize the size of a prediction set (efficiency), where this set is guaranteed to contain the true label with probability $\geq 1-\varepsilon$, for a user-defined $\varepsilon \in[0,1]$. CP works as a wrapper around a traditional learning model; yet, it is unclear how the prediction error of the underlying model affects the efficiency of the CP. In this note, we study a simple class of CPs whose efficiency is proportional to the prediction error of the underlying model.
APA
Cherubin, G.. (2023). How do the performance of a Conformal Predictor and its underlying algorithm relate?. Proceedings of the Twelfth Symposium on Conformal and Probabilistic Prediction with Applications, in Proceedings of Machine Learning Research 204:546-548 Available from https://proceedings.mlr.press/v204/cherubin23a.html.

Related Material