Shapley-value based inductive conformal prediction

William Lopez Jaramillo, Evgueni Smirnov
Proceedings of the Tenth Symposium on Conformal and Probabilistic Prediction and Applications, PMLR 152:52-71, 2021.

Abstract

Shapley values of individual instances were recently proposed for the problem of data valuation. They were defined as the average marginal instance contributions to the performance of a given predictor. In this paper we propose to use Shapley values of individual instances as conformity scores. To compute these values efficiently and exactly we employ a standard algorithm based on nearest neighbor classification and propose a variant of this algorithm for clustered data. Both variants are used for computing Shapley conformity scores for inductive conformal predictors. The experiments show that the Shapley-value conformity scores result in smaller prediction sets for significance level ϵ≤0.1 compared with those produced by standard conformity scores (i.e. similarity between true and predicted output values).

Cite this Paper

BibTeX
@InProceedings{pmlr-v152-jaramillo21a, title = {Shapley-value based inductive conformal prediction}, author = {Jaramillo, William Lopez and Smirnov, Evgueni}, booktitle = {Proceedings of the Tenth Symposium on Conformal and Probabilistic Prediction and Applications}, pages = {52--71}, year = {2021}, editor = {Carlsson, Lars and Luo, Zhiyuan and Cherubin, Giovanni and An Nguyen, Khuong}, volume = {152}, series = {Proceedings of Machine Learning Research}, month = {08--10 Sep}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v152/jaramillo21a/jaramillo21a.pdf}, url = {https://proceedings.mlr.press/v152/jaramillo21a.html}, abstract = {Shapley values of individual instances were recently proposed for the problem of data valuation. They were defined as the average marginal instance contributions to the performance of a given predictor. In this paper we propose to use Shapley values of individual instances as conformity scores. To compute these values efficiently and exactly we employ a standard algorithm based on nearest neighbor classification and propose a variant of this algorithm for clustered data. Both variants are used for computing Shapley conformity scores for inductive conformal predictors. The experiments show that the Shapley-value conformity scores result in smaller prediction sets for significance level $\epsilon \leq 0.1$ compared with those produced by standard conformity scores (i.e. similarity between true and predicted output values).} }
Endnote
%0 Conference Paper %T Shapley-value based inductive conformal prediction %A William Lopez Jaramillo %A Evgueni Smirnov %B Proceedings of the Tenth Symposium on Conformal and Probabilistic Prediction and Applications %C Proceedings of Machine Learning Research %D 2021 %E Lars Carlsson %E Zhiyuan Luo %E Giovanni Cherubin %E Khuong An Nguyen %F pmlr-v152-jaramillo21a %I PMLR %P 52--71 %U https://proceedings.mlr.press/v152/jaramillo21a.html %V 152 %X Shapley values of individual instances were recently proposed for the problem of data valuation. They were defined as the average marginal instance contributions to the performance of a given predictor. In this paper we propose to use Shapley values of individual instances as conformity scores. To compute these values efficiently and exactly we employ a standard algorithm based on nearest neighbor classification and propose a variant of this algorithm for clustered data. Both variants are used for computing Shapley conformity scores for inductive conformal predictors. The experiments show that the Shapley-value conformity scores result in smaller prediction sets for significance level $\epsilon \leq 0.1$ compared with those produced by standard conformity scores (i.e. similarity between true and predicted output values).
APA
Jaramillo, W.L. & Smirnov, E.. (2021). Shapley-value based inductive conformal prediction. Proceedings of the Tenth Symposium on Conformal and Probabilistic Prediction and Applications, in Proceedings of Machine Learning Research 152:52-71 Available from https://proceedings.mlr.press/v152/jaramillo21a.html.

Related Material