Manifold Restricted Interventional Shapley Values

Muhammad Faaiz Taufiq, Patrick Blöbaum, Lenon Minorics
Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, PMLR 206:5079-5106, 2023.

Abstract

Shapley values are model-agnostic methods for explaining model predictions. Many commonly used methods of computing Shapley values, known as off-manifold methods, rely on model evaluations on out-of-distribution input samples. Consequently, explanations obtained are sensitive to model behaviour outside the data distribution, which may be irrelevant for all practical purposes. While on-manifold methods have been proposed which do not suffer from this problem, we show that such methods are overly dependent on the input data distribution, and therefore result in unintuitive and misleading explanations. To circumvent these problems, we propose ManifoldShap, which respects the model’s domain of validity by restricting model evaluations to the data manifold. We show, theoretically and empirically, that ManifoldShap is robust to off-manifold perturbations of the model and leads to more accurate and intuitive explanations than existing state-of-the-art Shapley methods.

Cite this Paper

BibTeX
@InProceedings{pmlr-v206-taufiq23a, title = {Manifold Restricted Interventional Shapley Values}, author = {Taufiq, Muhammad Faaiz and Bl\"obaum, Patrick and Minorics, Lenon}, booktitle = {Proceedings of The 26th International Conference on Artificial Intelligence and Statistics}, pages = {5079--5106}, year = {2023}, editor = {Ruiz, Francisco and Dy, Jennifer and van de Meent, Jan-Willem}, volume = {206}, series = {Proceedings of Machine Learning Research}, month = {25--27 Apr}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v206/taufiq23a/taufiq23a.pdf}, url = {https://proceedings.mlr.press/v206/taufiq23a.html}, abstract = {Shapley values are model-agnostic methods for explaining model predictions. Many commonly used methods of computing Shapley values, known as off-manifold methods, rely on model evaluations on out-of-distribution input samples. Consequently, explanations obtained are sensitive to model behaviour outside the data distribution, which may be irrelevant for all practical purposes. While on-manifold methods have been proposed which do not suffer from this problem, we show that such methods are overly dependent on the input data distribution, and therefore result in unintuitive and misleading explanations. To circumvent these problems, we propose ManifoldShap, which respects the model’s domain of validity by restricting model evaluations to the data manifold. We show, theoretically and empirically, that ManifoldShap is robust to off-manifold perturbations of the model and leads to more accurate and intuitive explanations than existing state-of-the-art Shapley methods.} }
Endnote
%0 Conference Paper %T Manifold Restricted Interventional Shapley Values %A Muhammad Faaiz Taufiq %A Patrick Blöbaum %A Lenon Minorics %B Proceedings of The 26th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2023 %E Francisco Ruiz %E Jennifer Dy %E Jan-Willem van de Meent %F pmlr-v206-taufiq23a %I PMLR %P 5079--5106 %U https://proceedings.mlr.press/v206/taufiq23a.html %V 206 %X Shapley values are model-agnostic methods for explaining model predictions. Many commonly used methods of computing Shapley values, known as off-manifold methods, rely on model evaluations on out-of-distribution input samples. Consequently, explanations obtained are sensitive to model behaviour outside the data distribution, which may be irrelevant for all practical purposes. While on-manifold methods have been proposed which do not suffer from this problem, we show that such methods are overly dependent on the input data distribution, and therefore result in unintuitive and misleading explanations. To circumvent these problems, we propose ManifoldShap, which respects the model’s domain of validity by restricting model evaluations to the data manifold. We show, theoretically and empirically, that ManifoldShap is robust to off-manifold perturbations of the model and leads to more accurate and intuitive explanations than existing state-of-the-art Shapley methods.
APA
Taufiq, M.F., Blöbaum, P. & Minorics, L.. (2023). Manifold Restricted Interventional Shapley Values. Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 206:5079-5106 Available from https://proceedings.mlr.press/v206/taufiq23a.html.

Related Material