Fair Conformal Prediction for Individuals and Subgroups in Natural and Adversarial Settings: Theoretical Guarantees and Impossibility Results

Alberto Carlevaro, Luca Oneto, Massimo Massa, Davide Anguita, Fabio Roli
Proceedings of the Fifteenth Symposium on Conformal and Probabilistic Prediction with Applications, PMLR 329:761-792, 2026.

Abstract

Conformal prediction offers a principled, model-agnostic framework for constructing prediction sets with formal coverage guarantees under the assumption of exchangeability. In many practical applications, however, coverage alone is insufficient, and additional properties of the resulting prediction sets are often required. For instance, recent work has explored how to ensure CP fair treatment of both individuals and population subgroups and robustness to adversarial perturbations. In this paper, we study whether it is possible to construct conformal prediction sets that preserve coverage guarantees while simultaneously ensuring fair treatment of individuals, in the sense of counterfactual fairness by means of Equal Set Size, and of population subgroups, in the sense of Equalized Coverage and Equalized Average Set Size, in both natural and adversarial settings. In the adversarial setting, we consider an attacker whose goal is to induce unfair behavior toward either individual subjects or population subgroups, and we develop the first attack and defense methods for this scenario. We analyze both the case in which the sensitive attribute is available at test time and the more realistic setting in which it is not. For the natural setting, we propose conformal prediction methods that are provably efficient and satisfy fairness guarantees, and we also establish corresponding impossibility results. For the adversarial setting, we show that, under a realistic threat model in which the adversarial strategy is explicitly specified, the guarantees achieved in the natural setting can be recovered. Finally, experiments on real-world classification datasets involving fairness-sensitive tasks demonstrate the effectiveness and practical relevance of our approach, while also highlighting the limitations of existing methods.

Cite this Paper

BibTeX
@InProceedings{pmlr-v329-carlevaro26a, title = {Fair Conformal Prediction for Individuals and Subgroups in Natural and Adversarial Settings: Theoretical Guarantees and Impossibility Results}, author = {Carlevaro, Alberto and Oneto, Luca and Massa, Massimo and Anguita, Davide and Roli, Fabio}, booktitle = {Proceedings of the Fifteenth Symposium on Conformal and Probabilistic Prediction with Applications}, pages = {761--792}, year = {2026}, editor = {Ahlberg, Ernst and Johansson, Ulf and Boström, Henrik and Carlevaro, Alberto and Hallberg Szabadváry, Johan and Carlsson, Lars}, volume = {329}, series = {Proceedings of Machine Learning Research}, month = {02--04 Sep}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v329/main/assets/carlevaro26a/carlevaro26a.pdf}, url = {https://proceedings.mlr.press/v329/carlevaro26a.html}, abstract = {Conformal prediction offers a principled, model-agnostic framework for constructing prediction sets with formal coverage guarantees under the assumption of exchangeability. In many practical applications, however, coverage alone is insufficient, and additional properties of the resulting prediction sets are often required. For instance, recent work has explored how to ensure CP fair treatment of both individuals and population subgroups and robustness to adversarial perturbations. In this paper, we study whether it is possible to construct conformal prediction sets that preserve coverage guarantees while simultaneously ensuring fair treatment of individuals, in the sense of counterfactual fairness by means of Equal Set Size, and of population subgroups, in the sense of Equalized Coverage and Equalized Average Set Size, in both natural and adversarial settings. In the adversarial setting, we consider an attacker whose goal is to induce unfair behavior toward either individual subjects or population subgroups, and we develop the first attack and defense methods for this scenario. We analyze both the case in which the sensitive attribute is available at test time and the more realistic setting in which it is not. For the natural setting, we propose conformal prediction methods that are provably efficient and satisfy fairness guarantees, and we also establish corresponding impossibility results. For the adversarial setting, we show that, under a realistic threat model in which the adversarial strategy is explicitly specified, the guarantees achieved in the natural setting can be recovered. Finally, experiments on real-world classification datasets involving fairness-sensitive tasks demonstrate the effectiveness and practical relevance of our approach, while also highlighting the limitations of existing methods.} }
Endnote
%0 Conference Paper %T Fair Conformal Prediction for Individuals and Subgroups in Natural and Adversarial Settings: Theoretical Guarantees and Impossibility Results %A Alberto Carlevaro %A Luca Oneto %A Massimo Massa %A Davide Anguita %A Fabio Roli %B Proceedings of the Fifteenth Symposium on Conformal and Probabilistic Prediction with Applications %C Proceedings of Machine Learning Research %D 2026 %E Ernst Ahlberg %E Ulf Johansson %E Henrik Boström %E Alberto Carlevaro %E Johan Hallberg Szabadváry %E Lars Carlsson %F pmlr-v329-carlevaro26a %I PMLR %P 761--792 %U https://proceedings.mlr.press/v329/carlevaro26a.html %V 329 %X Conformal prediction offers a principled, model-agnostic framework for constructing prediction sets with formal coverage guarantees under the assumption of exchangeability. In many practical applications, however, coverage alone is insufficient, and additional properties of the resulting prediction sets are often required. For instance, recent work has explored how to ensure CP fair treatment of both individuals and population subgroups and robustness to adversarial perturbations. In this paper, we study whether it is possible to construct conformal prediction sets that preserve coverage guarantees while simultaneously ensuring fair treatment of individuals, in the sense of counterfactual fairness by means of Equal Set Size, and of population subgroups, in the sense of Equalized Coverage and Equalized Average Set Size, in both natural and adversarial settings. In the adversarial setting, we consider an attacker whose goal is to induce unfair behavior toward either individual subjects or population subgroups, and we develop the first attack and defense methods for this scenario. We analyze both the case in which the sensitive attribute is available at test time and the more realistic setting in which it is not. For the natural setting, we propose conformal prediction methods that are provably efficient and satisfy fairness guarantees, and we also establish corresponding impossibility results. For the adversarial setting, we show that, under a realistic threat model in which the adversarial strategy is explicitly specified, the guarantees achieved in the natural setting can be recovered. Finally, experiments on real-world classification datasets involving fairness-sensitive tasks demonstrate the effectiveness and practical relevance of our approach, while also highlighting the limitations of existing methods.
APA
Carlevaro, A., Oneto, L., Massa, M., Anguita, D. & Roli, F.. (2026). Fair Conformal Prediction for Individuals and Subgroups in Natural and Adversarial Settings: Theoretical Guarantees and Impossibility Results. Proceedings of the Fifteenth Symposium on Conformal and Probabilistic Prediction with Applications, in Proceedings of Machine Learning Research 329:761-792 Available from https://proceedings.mlr.press/v329/carlevaro26a.html.

Related Material