Adaptive Cumulative Mass Calibration with Conformal Prediction

Daniil Kazantsev, Eric Moulines, Maxim Panov, Nikita Kotelevskii, Mohsen Guizani
Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence, PMLR 337:2894-2916, 2026.

Abstract

Reliable probability estimates by classifiers are essential in high-risk applications. In practice, however, predicted probabilities are often miscalibrated, and many existing post-hoc calibration methods typically lack guarantees that a specific notion of calibration is achieved after the correction procedure is applied. We introduce a *set-based* perspective on calibration through the notion of *cumulative mass calibration* and the corresponding error measures. We propose a new calibration procedure based on conformal prediction that forms cumulative probabilities with guaranteed marginal coverage. We introduce an __adaptive temperature scaling algorithm__, with the temperature tuned for each input to satisfy the conformal coverage constraint. As we show, this procedure can be efficiently implemented. Across image classification tasks, particularly in settings with many classes, our method improves newly introduced calibration error measures (__CMCE__ and $\alpha$__-CMCE__) *and* standard metrics (such as {ECE}, cw-{ECE}, MCE) over the existing baselines.

Cite this Paper

BibTeX
@InProceedings{pmlr-v337-kazantsev26a, title = {Adaptive Cumulative Mass Calibration with Conformal Prediction}, author = {Kazantsev, Daniil and Moulines, Eric and Panov, Maxim and Kotelevskii, Nikita and Guizani, Mohsen}, booktitle = {Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence}, pages = {2894--2916}, year = {2026}, editor = {Perković, Emilija and Malinsky, Daniel}, volume = {337}, series = {Proceedings of Machine Learning Research}, month = {17--21 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v337/main/assets/kazantsev26a/kazantsev26a.pdf}, url = {https://proceedings.mlr.press/v337/kazantsev26a.html}, abstract = {Reliable probability estimates by classifiers are essential in high-risk applications. In practice, however, predicted probabilities are often miscalibrated, and many existing post-hoc calibration methods typically lack guarantees that a specific notion of calibration is achieved after the correction procedure is applied. We introduce a *set-based* perspective on calibration through the notion of *cumulative mass calibration* and the corresponding error measures. We propose a new calibration procedure based on conformal prediction that forms cumulative probabilities with guaranteed marginal coverage. We introduce an __adaptive temperature scaling algorithm__, with the temperature tuned for each input to satisfy the conformal coverage constraint. As we show, this procedure can be efficiently implemented. Across image classification tasks, particularly in settings with many classes, our method improves newly introduced calibration error measures (__CMCE__ and $\alpha$__-CMCE__) *and* standard metrics (such as {ECE}, cw-{ECE}, MCE) over the existing baselines.} }
Endnote
%0 Conference Paper %T Adaptive Cumulative Mass Calibration with Conformal Prediction %A Daniil Kazantsev %A Eric Moulines %A Maxim Panov %A Nikita Kotelevskii %A Mohsen Guizani %B Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2026 %E Emilija Perković %E Daniel Malinsky %F pmlr-v337-kazantsev26a %I PMLR %P 2894--2916 %U https://proceedings.mlr.press/v337/kazantsev26a.html %V 337 %X Reliable probability estimates by classifiers are essential in high-risk applications. In practice, however, predicted probabilities are often miscalibrated, and many existing post-hoc calibration methods typically lack guarantees that a specific notion of calibration is achieved after the correction procedure is applied. We introduce a *set-based* perspective on calibration through the notion of *cumulative mass calibration* and the corresponding error measures. We propose a new calibration procedure based on conformal prediction that forms cumulative probabilities with guaranteed marginal coverage. We introduce an __adaptive temperature scaling algorithm__, with the temperature tuned for each input to satisfy the conformal coverage constraint. As we show, this procedure can be efficiently implemented. Across image classification tasks, particularly in settings with many classes, our method improves newly introduced calibration error measures (__CMCE__ and $\alpha$__-CMCE__) *and* standard metrics (such as {ECE}, cw-{ECE}, MCE) over the existing baselines.
APA
Kazantsev, D., Moulines, E., Panov, M., Kotelevskii, N. & Guizani, M.. (2026). Adaptive Cumulative Mass Calibration with Conformal Prediction. Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research 337:2894-2916 Available from https://proceedings.mlr.press/v337/kazantsev26a.html.

Related Material