Explaining Probabilistic Models with Distributional Values

Luca Franceschi, Michele Donini, Cedric Archambeau, Matthias Seeger
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:13840-13863, 2024.

Abstract

A large branch of explainable machine learning is grounded in cooperative game theory. However, research indicates that game-theoretic explanations may mislead or be hard to interpret. We argue that often there is a critical mismatch between what one wishes to explain (e.g. the output of a classifier) and what current methods such as SHAP explain (e.g. the scalar probability of a class). This paper addresses such gap for probabilistic models by generalising cooperative games and value operators. We introduce the distributional values, random variables that track changes in the model output (e.g. flipping of the predicted class) and derive their analytic expressions for games with Gaussian, Bernoulli and Categorical payoffs. We further establish several characterising properties, and show that our framework provides fine-grained and insightful explanations with case studies on vision and language models.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-franceschi24a, title = {Explaining Probabilistic Models with Distributional Values}, author = {Franceschi, Luca and Donini, Michele and Archambeau, Cedric and Seeger, Matthias}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {13840--13863}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/franceschi24a/franceschi24a.pdf}, url = {https://proceedings.mlr.press/v235/franceschi24a.html}, abstract = {A large branch of explainable machine learning is grounded in cooperative game theory. However, research indicates that game-theoretic explanations may mislead or be hard to interpret. We argue that often there is a critical mismatch between what one wishes to explain (e.g. the output of a classifier) and what current methods such as SHAP explain (e.g. the scalar probability of a class). This paper addresses such gap for probabilistic models by generalising cooperative games and value operators. We introduce the distributional values, random variables that track changes in the model output (e.g. flipping of the predicted class) and derive their analytic expressions for games with Gaussian, Bernoulli and Categorical payoffs. We further establish several characterising properties, and show that our framework provides fine-grained and insightful explanations with case studies on vision and language models.} }
Endnote
%0 Conference Paper %T Explaining Probabilistic Models with Distributional Values %A Luca Franceschi %A Michele Donini %A Cedric Archambeau %A Matthias Seeger %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-franceschi24a %I PMLR %P 13840--13863 %U https://proceedings.mlr.press/v235/franceschi24a.html %V 235 %X A large branch of explainable machine learning is grounded in cooperative game theory. However, research indicates that game-theoretic explanations may mislead or be hard to interpret. We argue that often there is a critical mismatch between what one wishes to explain (e.g. the output of a classifier) and what current methods such as SHAP explain (e.g. the scalar probability of a class). This paper addresses such gap for probabilistic models by generalising cooperative games and value operators. We introduce the distributional values, random variables that track changes in the model output (e.g. flipping of the predicted class) and derive their analytic expressions for games with Gaussian, Bernoulli and Categorical payoffs. We further establish several characterising properties, and show that our framework provides fine-grained and insightful explanations with case studies on vision and language models.
APA
Franceschi, L., Donini, M., Archambeau, C. & Seeger, M.. (2024). Explaining Probabilistic Models with Distributional Values. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:13840-13863 Available from https://proceedings.mlr.press/v235/franceschi24a.html.

Related Material