Conformal prediction as a solution to distribution shifts in medical image analysis

Sol Erika Boman, Nita Mulliqi, Luana Xuan Liu, Kimmo Kartasalo, Martin Eklund
Proceedings of the Fifteenth Symposium on Conformal and Probabilistic Prediction with Applications, PMLR 329:1099-1102, 2026.

Abstract

Before medical AI systems can be widely deployed in clinics, there is a need for uncertainty quantification and improved handling of distribution shifts. Conformal prediction (CP) is an uncertainty quantification method that is highly useful but requires datasets to exhibit exchangeability, something that is broken by distribution shifts. In this work, we employ CP on real-world pathology image data to investigate distribution shifts’ impact on the validity of CP. We find that the main driver for loss of exchangeability is disagreements between pathologists, rather than medical differences between patients.

Cite this Paper

BibTeX
@InProceedings{pmlr-v329-boman26a, title = {Conformal prediction as a solution to distribution shifts in medical image analysis}, author = {Boman, Sol Erika and Mulliqi, Nita and Liu, Luana Xuan and Kartasalo, Kimmo and Eklund, Martin}, booktitle = {Proceedings of the Fifteenth Symposium on Conformal and Probabilistic Prediction with Applications}, pages = {1099--1102}, year = {2026}, editor = {Ahlberg, Ernst and Johansson, Ulf and Boström, Henrik and Carlevaro, Alberto and Hallberg Szabadváry, Johan and Carlsson, Lars}, volume = {329}, series = {Proceedings of Machine Learning Research}, month = {02--04 Sep}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v329/main/assets/boman26a/boman26a.pdf}, url = {https://proceedings.mlr.press/v329/boman26a.html}, abstract = {Before medical AI systems can be widely deployed in clinics, there is a need for uncertainty quantification and improved handling of distribution shifts. Conformal prediction (CP) is an uncertainty quantification method that is highly useful but requires datasets to exhibit exchangeability, something that is broken by distribution shifts. In this work, we employ CP on real-world pathology image data to investigate distribution shifts’ impact on the validity of CP. We find that the main driver for loss of exchangeability is disagreements between pathologists, rather than medical differences between patients.} }
Endnote
%0 Conference Paper %T Conformal prediction as a solution to distribution shifts in medical image analysis %A Sol Erika Boman %A Nita Mulliqi %A Luana Xuan Liu %A Kimmo Kartasalo %A Martin Eklund %B Proceedings of the Fifteenth Symposium on Conformal and Probabilistic Prediction with Applications %C Proceedings of Machine Learning Research %D 2026 %E Ernst Ahlberg %E Ulf Johansson %E Henrik Boström %E Alberto Carlevaro %E Johan Hallberg Szabadváry %E Lars Carlsson %F pmlr-v329-boman26a %I PMLR %P 1099--1102 %U https://proceedings.mlr.press/v329/boman26a.html %V 329 %X Before medical AI systems can be widely deployed in clinics, there is a need for uncertainty quantification and improved handling of distribution shifts. Conformal prediction (CP) is an uncertainty quantification method that is highly useful but requires datasets to exhibit exchangeability, something that is broken by distribution shifts. In this work, we employ CP on real-world pathology image data to investigate distribution shifts’ impact on the validity of CP. We find that the main driver for loss of exchangeability is disagreements between pathologists, rather than medical differences between patients.
APA
Boman, S.E., Mulliqi, N., Liu, L.X., Kartasalo, K. & Eklund, M.. (2026). Conformal prediction as a solution to distribution shifts in medical image analysis. Proceedings of the Fifteenth Symposium on Conformal and Probabilistic Prediction with Applications, in Proceedings of Machine Learning Research 329:1099-1102 Available from https://proceedings.mlr.press/v329/boman26a.html.

Related Material