KernelSHAP-IQ: Weighted Least Square Optimization for Shapley Interactions

Fabian Fumagalli, Maximilian Muschalik, Patrick Kolpaczki, Eyke Hüllermeier, Barbara Hammer
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:14308-14342, 2024.

Abstract

The Shapley value (SV) is a prevalent approach of allocating credit to machine learning (ML) entities to understand black box ML models. Enriching such interpretations with higher-order interactions is inevitable for complex systems, where the Shapley Interaction Index (SII) is a direct axiomatic extension of the SV. While it is well-known that the SV yields an optimal approximation of any game via a weighted least square (WLS) objective, an extension of this result to SII has been a long-standing open problem, which even led to the proposal of an alternative index. In this work, we characterize higher-order SII as a solution to a WLS problem, which constructs an optimal approximation via SII and k-Shapley values (k-SII). We prove this representation for the SV and pairwise SII and give empirically validated conjectures for higher orders. As a result, we propose KernelSHAP-IQ, a direct extension of KernelSHAP for SII, and demonstrate state-of-the-art performance for feature interactions.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-fumagalli24a, title = {{K}ernel{SHAP}-{IQ}: Weighted Least Square Optimization for Shapley Interactions}, author = {Fumagalli, Fabian and Muschalik, Maximilian and Kolpaczki, Patrick and H\"{u}llermeier, Eyke and Hammer, Barbara}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {14308--14342}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/fumagalli24a/fumagalli24a.pdf}, url = {https://proceedings.mlr.press/v235/fumagalli24a.html}, abstract = {The Shapley value (SV) is a prevalent approach of allocating credit to machine learning (ML) entities to understand black box ML models. Enriching such interpretations with higher-order interactions is inevitable for complex systems, where the Shapley Interaction Index (SII) is a direct axiomatic extension of the SV. While it is well-known that the SV yields an optimal approximation of any game via a weighted least square (WLS) objective, an extension of this result to SII has been a long-standing open problem, which even led to the proposal of an alternative index. In this work, we characterize higher-order SII as a solution to a WLS problem, which constructs an optimal approximation via SII and k-Shapley values (k-SII). We prove this representation for the SV and pairwise SII and give empirically validated conjectures for higher orders. As a result, we propose KernelSHAP-IQ, a direct extension of KernelSHAP for SII, and demonstrate state-of-the-art performance for feature interactions.} }
Endnote
%0 Conference Paper %T KernelSHAP-IQ: Weighted Least Square Optimization for Shapley Interactions %A Fabian Fumagalli %A Maximilian Muschalik %A Patrick Kolpaczki %A Eyke Hüllermeier %A Barbara Hammer %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-fumagalli24a %I PMLR %P 14308--14342 %U https://proceedings.mlr.press/v235/fumagalli24a.html %V 235 %X The Shapley value (SV) is a prevalent approach of allocating credit to machine learning (ML) entities to understand black box ML models. Enriching such interpretations with higher-order interactions is inevitable for complex systems, where the Shapley Interaction Index (SII) is a direct axiomatic extension of the SV. While it is well-known that the SV yields an optimal approximation of any game via a weighted least square (WLS) objective, an extension of this result to SII has been a long-standing open problem, which even led to the proposal of an alternative index. In this work, we characterize higher-order SII as a solution to a WLS problem, which constructs an optimal approximation via SII and k-Shapley values (k-SII). We prove this representation for the SV and pairwise SII and give empirically validated conjectures for higher orders. As a result, we propose KernelSHAP-IQ, a direct extension of KernelSHAP for SII, and demonstrate state-of-the-art performance for feature interactions.
APA
Fumagalli, F., Muschalik, M., Kolpaczki, P., Hüllermeier, E. & Hammer, B.. (2024). KernelSHAP-IQ: Weighted Least Square Optimization for Shapley Interactions. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:14308-14342 Available from https://proceedings.mlr.press/v235/fumagalli24a.html.

Related Material