Explaining Conformal Prediction: Diagnosing Reliability through Feature-Conditioned Outcomes and p-Value Margins

Antonio Caparrini, Miller-Janny Ariza-Garzón, Javier Arroyo
Proceedings of the Fifteenth Symposium on Conformal and Probabilistic Prediction with Applications, PMLR 329:509-528, 2026.

Abstract

Conformal prediction produces prediction sets with finite-sample coverage guarantees, and its practical utility for decision-making grows when the reliability of its predictions can be explained at the feature level. Yet, the intersection of conformal prediction and explainability remains relatively underexplored, leaving practitioners without tools to interpret how input features shape prediction reliability. We introduce a framework for explaining the reliability of conformal classifiers at the feature level using two tools. First, Feature-Conditioned Outcome Distribution (FCOD) plots visualize how correct, incorrect, and ambiguous conformal prediction set outcomes vary across feature ranges, identifying regions of the feature space associated with different empirical behaviors in terms of correctness and uncertainty of the predictions. Second, we propose two metrics derived from class-conditional conformal p-values: the label-free evidence margin, which provides a signed contrast between competing classes without the true label, and the label-dependent correctness margin, which compares the conformal p-value of the true class with that of its strongest alternative. They are used as SHAP targets for local feature attribution and aggregated global analysis, enabling feature-level explanations of relative conformal evidence between competing classes and support for the true class.

Cite this Paper

BibTeX
@InProceedings{pmlr-v329-caparrini26a, title = {Explaining Conformal Prediction: Diagnosing Reliability through Feature-Conditioned Outcomes and p-Value Margins}, author = {Caparrini, Antonio and Ariza-Garz{\'o}n, Miller-Janny and Arroyo, Javier}, booktitle = {Proceedings of the Fifteenth Symposium on Conformal and Probabilistic Prediction with Applications}, pages = {509--528}, year = {2026}, editor = {Ahlberg, Ernst and Johansson, Ulf and Boström, Henrik and Carlevaro, Alberto and Hallberg Szabadváry, Johan and Carlsson, Lars}, volume = {329}, series = {Proceedings of Machine Learning Research}, month = {02--04 Sep}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v329/main/assets/caparrini26a/caparrini26a.pdf}, url = {https://proceedings.mlr.press/v329/caparrini26a.html}, abstract = {Conformal prediction produces prediction sets with finite-sample coverage guarantees, and its practical utility for decision-making grows when the reliability of its predictions can be explained at the feature level. Yet, the intersection of conformal prediction and explainability remains relatively underexplored, leaving practitioners without tools to interpret how input features shape prediction reliability. We introduce a framework for explaining the reliability of conformal classifiers at the feature level using two tools. First, Feature-Conditioned Outcome Distribution (FCOD) plots visualize how correct, incorrect, and ambiguous conformal prediction set outcomes vary across feature ranges, identifying regions of the feature space associated with different empirical behaviors in terms of correctness and uncertainty of the predictions. Second, we propose two metrics derived from class-conditional conformal p-values: the label-free evidence margin, which provides a signed contrast between competing classes without the true label, and the label-dependent correctness margin, which compares the conformal p-value of the true class with that of its strongest alternative. They are used as SHAP targets for local feature attribution and aggregated global analysis, enabling feature-level explanations of relative conformal evidence between competing classes and support for the true class.} }
Endnote
%0 Conference Paper %T Explaining Conformal Prediction: Diagnosing Reliability through Feature-Conditioned Outcomes and p-Value Margins %A Antonio Caparrini %A Miller-Janny Ariza-Garzón %A Javier Arroyo %B Proceedings of the Fifteenth Symposium on Conformal and Probabilistic Prediction with Applications %C Proceedings of Machine Learning Research %D 2026 %E Ernst Ahlberg %E Ulf Johansson %E Henrik Boström %E Alberto Carlevaro %E Johan Hallberg Szabadváry %E Lars Carlsson %F pmlr-v329-caparrini26a %I PMLR %P 509--528 %U https://proceedings.mlr.press/v329/caparrini26a.html %V 329 %X Conformal prediction produces prediction sets with finite-sample coverage guarantees, and its practical utility for decision-making grows when the reliability of its predictions can be explained at the feature level. Yet, the intersection of conformal prediction and explainability remains relatively underexplored, leaving practitioners without tools to interpret how input features shape prediction reliability. We introduce a framework for explaining the reliability of conformal classifiers at the feature level using two tools. First, Feature-Conditioned Outcome Distribution (FCOD) plots visualize how correct, incorrect, and ambiguous conformal prediction set outcomes vary across feature ranges, identifying regions of the feature space associated with different empirical behaviors in terms of correctness and uncertainty of the predictions. Second, we propose two metrics derived from class-conditional conformal p-values: the label-free evidence margin, which provides a signed contrast between competing classes without the true label, and the label-dependent correctness margin, which compares the conformal p-value of the true class with that of its strongest alternative. They are used as SHAP targets for local feature attribution and aggregated global analysis, enabling feature-level explanations of relative conformal evidence between competing classes and support for the true class.
APA
Caparrini, A., Ariza-Garzón, M. & Arroyo, J.. (2026). Explaining Conformal Prediction: Diagnosing Reliability through Feature-Conditioned Outcomes and p-Value Margins. Proceedings of the Fifteenth Symposium on Conformal and Probabilistic Prediction with Applications, in Proceedings of Machine Learning Research 329:509-528 Available from https://proceedings.mlr.press/v329/caparrini26a.html.

Related Material