Efficient Conformal Prediction under Data Heterogeneity

Vincent Plassier, Nikita Kotelevskii, Aleksandr Rubashevskii, Fedor Noskov, Maksim Velikanov, Alexander Fishkov, Samuel Horvath, Martin Takac, Eric Moulines, Maxim Panov
Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, PMLR 238:4879-4887, 2024.

Abstract

Conformal prediction (CP) stands out as a robust framework for uncertainty quantification, which is crucial for ensuring the reliability of predictions. However, common CP methods heavily rely on the data exchangeability, a condition often violated in practice. Existing approaches for tackling non-exchangeability lead to methods that are not computable beyond the simplest examples. In this work, we introduce a new efficient approach to CP that produces provably valid confidence sets for fairly general non-exchangeable data distributions. We illustrate the general theory with applications to the challenging setting of federated learning under data heterogeneity between agents. Our method allows constructing provably valid personalized prediction sets for agents in a fully federated way. The effectiveness of the proposed method is demonstrated in a series of experiments on real-world datasets.

Cite this Paper

BibTeX
@InProceedings{pmlr-v238-plassier24a, title = {Efficient Conformal Prediction under Data Heterogeneity}, author = {Plassier, Vincent and Kotelevskii, Nikita and Rubashevskii, Aleksandr and Noskov, Fedor and Velikanov, Maksim and Fishkov, Alexander and Horvath, Samuel and Takac, Martin and Moulines, Eric and Panov, Maxim}, booktitle = {Proceedings of The 27th International Conference on Artificial Intelligence and Statistics}, pages = {4879--4887}, year = {2024}, editor = {Dasgupta, Sanjoy and Mandt, Stephan and Li, Yingzhen}, volume = {238}, series = {Proceedings of Machine Learning Research}, month = {02--04 May}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v238/plassier24a/plassier24a.pdf}, url = {https://proceedings.mlr.press/v238/plassier24a.html}, abstract = {Conformal prediction (CP) stands out as a robust framework for uncertainty quantification, which is crucial for ensuring the reliability of predictions. However, common CP methods heavily rely on the data exchangeability, a condition often violated in practice. Existing approaches for tackling non-exchangeability lead to methods that are not computable beyond the simplest examples. In this work, we introduce a new efficient approach to CP that produces provably valid confidence sets for fairly general non-exchangeable data distributions. We illustrate the general theory with applications to the challenging setting of federated learning under data heterogeneity between agents. Our method allows constructing provably valid personalized prediction sets for agents in a fully federated way. The effectiveness of the proposed method is demonstrated in a series of experiments on real-world datasets.} }
Endnote
%0 Conference Paper %T Efficient Conformal Prediction under Data Heterogeneity %A Vincent Plassier %A Nikita Kotelevskii %A Aleksandr Rubashevskii %A Fedor Noskov %A Maksim Velikanov %A Alexander Fishkov %A Samuel Horvath %A Martin Takac %A Eric Moulines %A Maxim Panov %B Proceedings of The 27th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2024 %E Sanjoy Dasgupta %E Stephan Mandt %E Yingzhen Li %F pmlr-v238-plassier24a %I PMLR %P 4879--4887 %U https://proceedings.mlr.press/v238/plassier24a.html %V 238 %X Conformal prediction (CP) stands out as a robust framework for uncertainty quantification, which is crucial for ensuring the reliability of predictions. However, common CP methods heavily rely on the data exchangeability, a condition often violated in practice. Existing approaches for tackling non-exchangeability lead to methods that are not computable beyond the simplest examples. In this work, we introduce a new efficient approach to CP that produces provably valid confidence sets for fairly general non-exchangeable data distributions. We illustrate the general theory with applications to the challenging setting of federated learning under data heterogeneity between agents. Our method allows constructing provably valid personalized prediction sets for agents in a fully federated way. The effectiveness of the proposed method is demonstrated in a series of experiments on real-world datasets.
APA
Plassier, V., Kotelevskii, N., Rubashevskii, A., Noskov, F., Velikanov, M., Fishkov, A., Horvath, S., Takac, M., Moulines, E. & Panov, M.. (2024). Efficient Conformal Prediction under Data Heterogeneity. Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 238:4879-4887 Available from https://proceedings.mlr.press/v238/plassier24a.html.

Related Material