Conformal Prediction for Federated Uncertainty Quantification Under Label Shift

Vincent Plassier, Mehdi Makni, Aleksandr Rubashevskii, Eric Moulines, Maxim Panov
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:27907-27947, 2023.

Abstract

Federated Learning (FL) is a machine learning framework where many clients collaboratively train models while keeping the training data decentralized. Despite recent advances in FL, the uncertainty quantification topic (UQ) remains partially addressed. Among UQ methods, conformal prediction (CP) approaches provides distribution-free guarantees under minimal assumptions. We develop a new federated conformal prediction method based on quantile regression and take into account privacy constraints. This method takes advantage of importance weighting to effectively address the label shift between agents and provides theoretical guarantees for both valid coverage of the prediction sets and differential privacy. Extensive experimental studies demonstrate that this method outperforms current competitors.

Cite this Paper

BibTeX
@InProceedings{pmlr-v202-plassier23a, title = {Conformal Prediction for Federated Uncertainty Quantification Under Label Shift}, author = {Plassier, Vincent and Makni, Mehdi and Rubashevskii, Aleksandr and Moulines, Eric and Panov, Maxim}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {27907--27947}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/plassier23a/plassier23a.pdf}, url = {https://proceedings.mlr.press/v202/plassier23a.html}, abstract = {Federated Learning (FL) is a machine learning framework where many clients collaboratively train models while keeping the training data decentralized. Despite recent advances in FL, the uncertainty quantification topic (UQ) remains partially addressed. Among UQ methods, conformal prediction (CP) approaches provides distribution-free guarantees under minimal assumptions. We develop a new federated conformal prediction method based on quantile regression and take into account privacy constraints. This method takes advantage of importance weighting to effectively address the label shift between agents and provides theoretical guarantees for both valid coverage of the prediction sets and differential privacy. Extensive experimental studies demonstrate that this method outperforms current competitors.} }
Endnote
%0 Conference Paper %T Conformal Prediction for Federated Uncertainty Quantification Under Label Shift %A Vincent Plassier %A Mehdi Makni %A Aleksandr Rubashevskii %A Eric Moulines %A Maxim Panov %B Proceedings of the 40th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Andreas Krause %E Emma Brunskill %E Kyunghyun Cho %E Barbara Engelhardt %E Sivan Sabato %E Jonathan Scarlett %F pmlr-v202-plassier23a %I PMLR %P 27907--27947 %U https://proceedings.mlr.press/v202/plassier23a.html %V 202 %X Federated Learning (FL) is a machine learning framework where many clients collaboratively train models while keeping the training data decentralized. Despite recent advances in FL, the uncertainty quantification topic (UQ) remains partially addressed. Among UQ methods, conformal prediction (CP) approaches provides distribution-free guarantees under minimal assumptions. We develop a new federated conformal prediction method based on quantile regression and take into account privacy constraints. This method takes advantage of importance weighting to effectively address the label shift between agents and provides theoretical guarantees for both valid coverage of the prediction sets and differential privacy. Extensive experimental studies demonstrate that this method outperforms current competitors.
APA
Plassier, V., Makni, M., Rubashevskii, A., Moulines, E. & Panov, M.. (2023). Conformal Prediction for Federated Uncertainty Quantification Under Label Shift. Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:27907-27947 Available from https://proceedings.mlr.press/v202/plassier23a.html.

Related Material