Valid and Efficient Uncertainty Quantification for Federated Joint Shift

Yuanjie Shi, Peihong Li, Xuanyu Cao, Yan Yan
Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence, PMLR 337:6232-6260, 2026.

Abstract

Reliable uncertainty quantification (UQ) is critical for safety-sensitive federated learning (FL) applications, such as cross-hospital diagnosis and global sensor networks. In FL, privacy constraints prevent centralized data pooling, while client data exhibit joint distribution shifts across covariates, labels, and conditionals. These shifts violate the exchangeability assumption required by conformal prediction ({CP}), which otherwise guarantees distribution-free coverage under i.i.d. data. To address this gap, we propose Federated Conformal Prediction for Joint Shift (FCPJS), enabling valid {CP} under heterogeneous clients without sharing raw data. Specifically, the method operates in two stages: (i) the server constructs a privacy-preserving global sketch of nonconformity score distributions; (ii) each client performs importance-weighted calibration by contrasting its local scores with the global sketch. This distribution-ratio weighting corrects joint shifts in a unified manner. We prove that FCPJS attains valid marginal coverage across clients, up to an $O(\sqrt{\varepsilon_m}+1/\sqrt{n})$ error from the finite number of calibration samples $n$ and sketch size $m$. Experiments on four heterogeneous benchmarks show that \newCP{} preserves coverage and improves predictive efficiency by $12.64%$ over the strongest baseline. To our knowledge, FCPJS is the first method providing provably valid and efficient conformal UQ for FL under joint distribution shift.

Cite this Paper

BibTeX
@InProceedings{pmlr-v337-shi26a, title = {Valid and Efficient Uncertainty Quantification for Federated Joint Shift}, author = {Shi, Yuanjie and Li, Peihong and Cao, Xuanyu and Yan, Yan}, booktitle = {Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence}, pages = {6232--6260}, year = {2026}, editor = {Perković, Emilija and Malinsky, Daniel}, volume = {337}, series = {Proceedings of Machine Learning Research}, month = {17--21 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v337/main/assets/shi26a/shi26a.pdf}, url = {https://proceedings.mlr.press/v337/shi26a.html}, abstract = {Reliable uncertainty quantification (UQ) is critical for safety-sensitive federated learning (FL) applications, such as cross-hospital diagnosis and global sensor networks. In FL, privacy constraints prevent centralized data pooling, while client data exhibit joint distribution shifts across covariates, labels, and conditionals. These shifts violate the exchangeability assumption required by conformal prediction ({CP}), which otherwise guarantees distribution-free coverage under i.i.d. data. To address this gap, we propose Federated Conformal Prediction for Joint Shift (FCPJS), enabling valid {CP} under heterogeneous clients without sharing raw data. Specifically, the method operates in two stages: (i) the server constructs a privacy-preserving global sketch of nonconformity score distributions; (ii) each client performs importance-weighted calibration by contrasting its local scores with the global sketch. This distribution-ratio weighting corrects joint shifts in a unified manner. We prove that FCPJS attains valid marginal coverage across clients, up to an $O(\sqrt{\varepsilon_m}+1/\sqrt{n})$ error from the finite number of calibration samples $n$ and sketch size $m$. Experiments on four heterogeneous benchmarks show that \newCP{} preserves coverage and improves predictive efficiency by $12.64%$ over the strongest baseline. To our knowledge, FCPJS is the first method providing provably valid and efficient conformal UQ for FL under joint distribution shift.} }
Endnote
%0 Conference Paper %T Valid and Efficient Uncertainty Quantification for Federated Joint Shift %A Yuanjie Shi %A Peihong Li %A Xuanyu Cao %A Yan Yan %B Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2026 %E Emilija Perković %E Daniel Malinsky %F pmlr-v337-shi26a %I PMLR %P 6232--6260 %U https://proceedings.mlr.press/v337/shi26a.html %V 337 %X Reliable uncertainty quantification (UQ) is critical for safety-sensitive federated learning (FL) applications, such as cross-hospital diagnosis and global sensor networks. In FL, privacy constraints prevent centralized data pooling, while client data exhibit joint distribution shifts across covariates, labels, and conditionals. These shifts violate the exchangeability assumption required by conformal prediction ({CP}), which otherwise guarantees distribution-free coverage under i.i.d. data. To address this gap, we propose Federated Conformal Prediction for Joint Shift (FCPJS), enabling valid {CP} under heterogeneous clients without sharing raw data. Specifically, the method operates in two stages: (i) the server constructs a privacy-preserving global sketch of nonconformity score distributions; (ii) each client performs importance-weighted calibration by contrasting its local scores with the global sketch. This distribution-ratio weighting corrects joint shifts in a unified manner. We prove that FCPJS attains valid marginal coverage across clients, up to an $O(\sqrt{\varepsilon_m}+1/\sqrt{n})$ error from the finite number of calibration samples $n$ and sketch size $m$. Experiments on four heterogeneous benchmarks show that \newCP{} preserves coverage and improves predictive efficiency by $12.64%$ over the strongest baseline. To our knowledge, FCPJS is the first method providing provably valid and efficient conformal UQ for FL under joint distribution shift.
APA
Shi, Y., Li, P., Cao, X. & Yan, Y.. (2026). Valid and Efficient Uncertainty Quantification for Federated Joint Shift. Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research 337:6232-6260 Available from https://proceedings.mlr.press/v337/shi26a.html.

Related Material