PUNCC: a Python Library for Predictive Uncertainty Calibration and Conformalization

Mouhcine Mendil, Luca Mossina, David Vigouroux
Proceedings of the Twelfth Symposium on Conformal and Probabilistic Prediction with Applications, PMLR 204:582-601, 2023.

Abstract

Predictive UNcertainty Calibration and Conformalization (PUNCC) is an open-source Python library integrating a collection of state-of-the-art Conformal Prediction (CP) algorithms and related techniques for regression and classification problems. This package aims to make conformal procedures accessible to non-experts using a simple and intuitive implementation. It is compatible with scikit-learn, PyTorch and TensorFlow and easily extensible to other prediction toolkits. PUNCC also comes with a low-level API that provides a unfified workfow in a pythonic environment to build, combine and run inductive CP algorithms. It offers generic structures and consistent interfaces to design customized nonconformity scores, data partition schemes, and methods for constructing prediction sets. In this paper, we present the design of our library and demonstrate its use with various CP procedures, Machine Learning (ML) problems and models from different ML libraries. Source code, documentation and demos are available at https://github.com/deel-ai/puncc.

Cite this Paper

BibTeX
@InProceedings{pmlr-v204-mendil23a, title = {PUNCC: a Python Library for Predictive Uncertainty Calibration and Conformalization}, author = {Mendil, Mouhcine and Mossina, Luca and Vigouroux, David}, booktitle = {Proceedings of the Twelfth Symposium on Conformal and Probabilistic Prediction with Applications}, pages = {582--601}, year = {2023}, editor = {Papadopoulos, Harris and Nguyen, Khuong An and Boström, Henrik and Carlsson, Lars}, volume = {204}, series = {Proceedings of Machine Learning Research}, month = {13--15 Sep}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v204/mendil23a/mendil23a.pdf}, url = {https://proceedings.mlr.press/v204/mendil23a.html}, abstract = {Predictive UNcertainty Calibration and Conformalization (PUNCC) is an open-source Python library integrating a collection of state-of-the-art Conformal Prediction (CP) algorithms and related techniques for regression and classification problems. This package aims to make conformal procedures accessible to non-experts using a simple and intuitive implementation. It is compatible with scikit-learn, PyTorch and TensorFlow and easily extensible to other prediction toolkits. PUNCC also comes with a low-level API that provides a unfified workfow in a pythonic environment to build, combine and run inductive CP algorithms. It offers generic structures and consistent interfaces to design customized nonconformity scores, data partition schemes, and methods for constructing prediction sets. In this paper, we present the design of our library and demonstrate its use with various CP procedures, Machine Learning (ML) problems and models from different ML libraries. Source code, documentation and demos are available at https://github.com/deel-ai/puncc.} }
Endnote
%0 Conference Paper %T PUNCC: a Python Library for Predictive Uncertainty Calibration and Conformalization %A Mouhcine Mendil %A Luca Mossina %A David Vigouroux %B Proceedings of the Twelfth Symposium on Conformal and Probabilistic Prediction with Applications %C Proceedings of Machine Learning Research %D 2023 %E Harris Papadopoulos %E Khuong An Nguyen %E Henrik Boström %E Lars Carlsson %F pmlr-v204-mendil23a %I PMLR %P 582--601 %U https://proceedings.mlr.press/v204/mendil23a.html %V 204 %X Predictive UNcertainty Calibration and Conformalization (PUNCC) is an open-source Python library integrating a collection of state-of-the-art Conformal Prediction (CP) algorithms and related techniques for regression and classification problems. This package aims to make conformal procedures accessible to non-experts using a simple and intuitive implementation. It is compatible with scikit-learn, PyTorch and TensorFlow and easily extensible to other prediction toolkits. PUNCC also comes with a low-level API that provides a unfified workfow in a pythonic environment to build, combine and run inductive CP algorithms. It offers generic structures and consistent interfaces to design customized nonconformity scores, data partition schemes, and methods for constructing prediction sets. In this paper, we present the design of our library and demonstrate its use with various CP procedures, Machine Learning (ML) problems and models from different ML libraries. Source code, documentation and demos are available at https://github.com/deel-ai/puncc.
APA
Mendil, M., Mossina, L. & Vigouroux, D.. (2023). PUNCC: a Python Library for Predictive Uncertainty Calibration and Conformalization. Proceedings of the Twelfth Symposium on Conformal and Probabilistic Prediction with Applications, in Proceedings of Machine Learning Research 204:582-601 Available from https://proceedings.mlr.press/v204/mendil23a.html.

Related Material