A probabilistic scaling approach to conformal predictions in binary image classification

Alberto Carlevaro, Sara Narteni, Fabrizio Dabbene, Teodoro Alamo, Maurizio Mongelli
Proceedings of the Thirteenth Symposium on Conformal and Probabilistic Prediction with Applications, PMLR 230:28-43, 2024.

Abstract

Deep learning solutions for image classification are more and more widespread and sophisticated today, bringing the necessity to properly address their reliability. Many approaches exist in uncertainty quantification, and, among these, conformal prediction is one of most solid and well-established frameworks. In this paper, we study another approach, defined as deep probabilistic scaling, based on the notion of scalable classifiers, combined with probabilistic scaling from order statistics. Given a pre-trained neural network for (binary) image classification and a target class on which it is desirable to control the error, this method is able to bound that error to a user-defined level ($\varepsilon$). The method individuates probabilistic safety regions of target class samples correctly predicted in high probability. We show how the proposed method links with conformal prediction, discussing analogies and differences. By considering a (binary) convolutional neural network classifier, experiments on several benchmark datasets show a good overall performance of the methodology in controlling false negatives.

Cite this Paper

BibTeX
@InProceedings{pmlr-v230-carlevaro24a, title = {A probabilistic scaling approach to conformal predictions in binary image classification}, author = {Carlevaro, Alberto and Narteni, Sara and Dabbene, Fabrizio and Alamo, Teodoro and Mongelli, Maurizio}, booktitle = {Proceedings of the Thirteenth Symposium on Conformal and Probabilistic Prediction with Applications}, pages = {28--43}, year = {2024}, editor = {Vantini, Simone and Fontana, Matteo and Solari, Aldo and Boström, Henrik and Carlsson, Lars}, volume = {230}, series = {Proceedings of Machine Learning Research}, month = {09--11 Sep}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v230/main/assets/carlevaro24a/carlevaro24a.pdf}, url = {https://proceedings.mlr.press/v230/carlevaro24a.html}, abstract = {Deep learning solutions for image classification are more and more widespread and sophisticated today, bringing the necessity to properly address their reliability. Many approaches exist in uncertainty quantification, and, among these, conformal prediction is one of most solid and well-established frameworks. In this paper, we study another approach, defined as deep probabilistic scaling, based on the notion of scalable classifiers, combined with probabilistic scaling from order statistics. Given a pre-trained neural network for (binary) image classification and a target class on which it is desirable to control the error, this method is able to bound that error to a user-defined level ($\varepsilon$). The method individuates probabilistic safety regions of target class samples correctly predicted in high probability. We show how the proposed method links with conformal prediction, discussing analogies and differences. By considering a (binary) convolutional neural network classifier, experiments on several benchmark datasets show a good overall performance of the methodology in controlling false negatives.} }
Endnote
%0 Conference Paper %T A probabilistic scaling approach to conformal predictions in binary image classification %A Alberto Carlevaro %A Sara Narteni %A Fabrizio Dabbene %A Teodoro Alamo %A Maurizio Mongelli %B Proceedings of the Thirteenth Symposium on Conformal and Probabilistic Prediction with Applications %C Proceedings of Machine Learning Research %D 2024 %E Simone Vantini %E Matteo Fontana %E Aldo Solari %E Henrik Boström %E Lars Carlsson %F pmlr-v230-carlevaro24a %I PMLR %P 28--43 %U https://proceedings.mlr.press/v230/carlevaro24a.html %V 230 %X Deep learning solutions for image classification are more and more widespread and sophisticated today, bringing the necessity to properly address their reliability. Many approaches exist in uncertainty quantification, and, among these, conformal prediction is one of most solid and well-established frameworks. In this paper, we study another approach, defined as deep probabilistic scaling, based on the notion of scalable classifiers, combined with probabilistic scaling from order statistics. Given a pre-trained neural network for (binary) image classification and a target class on which it is desirable to control the error, this method is able to bound that error to a user-defined level ($\varepsilon$). The method individuates probabilistic safety regions of target class samples correctly predicted in high probability. We show how the proposed method links with conformal prediction, discussing analogies and differences. By considering a (binary) convolutional neural network classifier, experiments on several benchmark datasets show a good overall performance of the methodology in controlling false negatives.
APA
Carlevaro, A., Narteni, S., Dabbene, F., Alamo, T. & Mongelli, M.. (2024). A probabilistic scaling approach to conformal predictions in binary image classification. Proceedings of the Thirteenth Symposium on Conformal and Probabilistic Prediction with Applications, in Proceedings of Machine Learning Research 230:28-43 Available from https://proceedings.mlr.press/v230/carlevaro24a.html.

Related Material