online-cp: a Python Package for Online Conformal Prediction, Conformal Predictive Systems and Conformal Test Martingales

Johan Hallberg Szabadváry, Tuwe Löfström, Rudy Matela
Proceedings of the Fourteenth Symposium on Conformal and Probabilistic Prediction with Applications, PMLR 266:595-614, 2025.

Abstract

Conformal prediction (CP) has gained increasing attention in machine learning owing to its ability to provide reliable prediction sets with well-calibrated uncertainty estimates. While most existing CP implementations focus on inductive conformal prediction (ICP), full conformal prediction–also known as online or transductive CP–offers the strongest validity guarantees but has been largely absent from open-source software due to its computational complexity. In this paper, we introduce \texttt{online-cp}, a Python package designed for online conformal prediction, conformal predictive systems (CPS), and conformal test martingales. The package implements several online CP algorithms, enabling efficient and principled uncertainty quantification in streaming data scenarios. Additionally, it includes tools for testing the exchangeability assumption by using conformal test martingales. We demonstrate the functionality of \texttt{online-cp} through classification and regression examples as well as applications to predictive systems and exchangeability testing. By making online CP methods accessible, \texttt{online-cp} provides a foundation for broader adoption and further development of conformal prediction in real-time machine learning applications

Cite this Paper

BibTeX
@InProceedings{pmlr-v266-hallberg-szabadvary25a, title = {online-cp: a Python Package for Online Conformal Prediction, Conformal Predictive Systems and Conformal Test Martingales}, author = {Hallberg Szabadv\'{a}ry, Johan and L\"{o}fstr\"{o}m, Tuwe and Matela, Rudy}, booktitle = {Proceedings of the Fourteenth Symposium on Conformal and Probabilistic Prediction with Applications}, pages = {595--614}, year = {2025}, editor = {Nguyen, Khuong An and Luo, Zhiyuan and Papadopoulos, Harris and Löfström, Tuwe and Carlsson, Lars and Boström, Henrik}, volume = {266}, series = {Proceedings of Machine Learning Research}, month = {10--12 Sep}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v266/main/assets/hallberg-szabadvary25a/hallberg-szabadvary25a.pdf}, url = {https://proceedings.mlr.press/v266/hallberg-szabadvary25a.html}, abstract = {Conformal prediction (CP) has gained increasing attention in machine learning owing to its ability to provide reliable prediction sets with well-calibrated uncertainty estimates. While most existing CP implementations focus on inductive conformal prediction (ICP), full conformal prediction–also known as online or transductive CP–offers the strongest validity guarantees but has been largely absent from open-source software due to its computational complexity. In this paper, we introduce \texttt{online-cp}, a Python package designed for online conformal prediction, conformal predictive systems (CPS), and conformal test martingales. The package implements several online CP algorithms, enabling efficient and principled uncertainty quantification in streaming data scenarios. Additionally, it includes tools for testing the exchangeability assumption by using conformal test martingales. We demonstrate the functionality of \texttt{online-cp} through classification and regression examples as well as applications to predictive systems and exchangeability testing. By making online CP methods accessible, \texttt{online-cp} provides a foundation for broader adoption and further development of conformal prediction in real-time machine learning applications} }
Endnote
%0 Conference Paper %T online-cp: a Python Package for Online Conformal Prediction, Conformal Predictive Systems and Conformal Test Martingales %A Johan Hallberg Szabadváry %A Tuwe Löfström %A Rudy Matela %B Proceedings of the Fourteenth Symposium on Conformal and Probabilistic Prediction with Applications %C Proceedings of Machine Learning Research %D 2025 %E Khuong An Nguyen %E Zhiyuan Luo %E Harris Papadopoulos %E Tuwe Löfström %E Lars Carlsson %E Henrik Boström %F pmlr-v266-hallberg-szabadvary25a %I PMLR %P 595--614 %U https://proceedings.mlr.press/v266/hallberg-szabadvary25a.html %V 266 %X Conformal prediction (CP) has gained increasing attention in machine learning owing to its ability to provide reliable prediction sets with well-calibrated uncertainty estimates. While most existing CP implementations focus on inductive conformal prediction (ICP), full conformal prediction–also known as online or transductive CP–offers the strongest validity guarantees but has been largely absent from open-source software due to its computational complexity. In this paper, we introduce \texttt{online-cp}, a Python package designed for online conformal prediction, conformal predictive systems (CPS), and conformal test martingales. The package implements several online CP algorithms, enabling efficient and principled uncertainty quantification in streaming data scenarios. Additionally, it includes tools for testing the exchangeability assumption by using conformal test martingales. We demonstrate the functionality of \texttt{online-cp} through classification and regression examples as well as applications to predictive systems and exchangeability testing. By making online CP methods accessible, \texttt{online-cp} provides a foundation for broader adoption and further development of conformal prediction in real-time machine learning applications
APA
Hallberg Szabadváry, J., Löfström, T. & Matela, R.. (2025). online-cp: a Python Package for Online Conformal Prediction, Conformal Predictive Systems and Conformal Test Martingales. Proceedings of the Fourteenth Symposium on Conformal and Probabilistic Prediction with Applications, in Proceedings of Machine Learning Research 266:595-614 Available from https://proceedings.mlr.press/v266/hallberg-szabadvary25a.html.

Related Material