Feature relevance quantification in explainable AI: A causal problem

Dominik Janzing, Lenon Minorics, Patrick Bloebaum
Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, PMLR 108:2907-2916, 2020.

Abstract

We discuss promising recent contributions on quantifying feature relevance using Shapley values, where we observed some confusion on which probability distribution is the right one for dropped features. We argue that the confusion is based on not carefully distinguishing between observational and interventional conditional probabilities and try a clarification based on Pearl’s seminal work on causality. We conclude that unconditional rather than conditional expectations provide the right notion of dropping features. This contradicts the view of the authors of the software package SHAP. In that work, unconditional expectations (which we argue to be conceptually right) are only used as approximation for the conditional ones, which encouraged others to ’improve’ SHAP in a way that we believe to be flawed.

Cite this Paper

BibTeX
@InProceedings{pmlr-v108-janzing20a, title = {Feature relevance quantification in explainable AI: A causal problem}, author = {Janzing, Dominik and Minorics, Lenon and Bloebaum, Patrick}, booktitle = {Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics}, pages = {2907--2916}, year = {2020}, editor = {Chiappa, Silvia and Calandra, Roberto}, volume = {108}, series = {Proceedings of Machine Learning Research}, month = {26--28 Aug}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v108/janzing20a/janzing20a.pdf}, url = {https://proceedings.mlr.press/v108/janzing20a.html}, abstract = {We discuss promising recent contributions on quantifying feature relevance using Shapley values, where we observed some confusion on which probability distribution is the right one for dropped features. We argue that the confusion is based on not carefully distinguishing between observational and interventional conditional probabilities and try a clarification based on Pearl’s seminal work on causality. We conclude that unconditional rather than conditional expectations provide the right notion of dropping features. This contradicts the view of the authors of the software package SHAP. In that work, unconditional expectations (which we argue to be conceptually right) are only used as approximation for the conditional ones, which encouraged others to ’improve’ SHAP in a way that we believe to be flawed.} }
Endnote
%0 Conference Paper %T Feature relevance quantification in explainable AI: A causal problem %A Dominik Janzing %A Lenon Minorics %A Patrick Bloebaum %B Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2020 %E Silvia Chiappa %E Roberto Calandra %F pmlr-v108-janzing20a %I PMLR %P 2907--2916 %U https://proceedings.mlr.press/v108/janzing20a.html %V 108 %X We discuss promising recent contributions on quantifying feature relevance using Shapley values, where we observed some confusion on which probability distribution is the right one for dropped features. We argue that the confusion is based on not carefully distinguishing between observational and interventional conditional probabilities and try a clarification based on Pearl’s seminal work on causality. We conclude that unconditional rather than conditional expectations provide the right notion of dropping features. This contradicts the view of the authors of the software package SHAP. In that work, unconditional expectations (which we argue to be conceptually right) are only used as approximation for the conditional ones, which encouraged others to ’improve’ SHAP in a way that we believe to be flawed.
APA
Janzing, D., Minorics, L. & Bloebaum, P.. (2020). Feature relevance quantification in explainable AI: A causal problem. Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 108:2907-2916 Available from https://proceedings.mlr.press/v108/janzing20a.html.

Related Material