Sanity Simulations for Saliency Methods

Joon Sik Kim, Gregory Plumb, Ameet Talwalkar
Proceedings of the 39th International Conference on Machine Learning, PMLR 162:11173-11200, 2022.

Abstract

Saliency methods are a popular class of feature attribution explanation methods that aim to capture a model’s predictive reasoning by identifying "important" pixels in an input image. However, the development and adoption of these methods are hindered by the lack of access to ground-truth model reasoning, which prevents accurate evaluation. In this work, we design a synthetic benchmarking framework, SMERF, that allows us to perform ground-truth-based evaluation while controlling the complexity of the model’s reasoning. Experimentally, SMERF reveals significant limitations in existing saliency methods and, as a result, represents a useful tool for the development of new saliency methods.

Cite this Paper

BibTeX
@InProceedings{pmlr-v162-kim22h, title = {Sanity Simulations for Saliency Methods}, author = {Kim, Joon Sik and Plumb, Gregory and Talwalkar, Ameet}, booktitle = {Proceedings of the 39th International Conference on Machine Learning}, pages = {11173--11200}, year = {2022}, editor = {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan}, volume = {162}, series = {Proceedings of Machine Learning Research}, month = {17--23 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v162/kim22h/kim22h.pdf}, url = {https://proceedings.mlr.press/v162/kim22h.html}, abstract = {Saliency methods are a popular class of feature attribution explanation methods that aim to capture a model’s predictive reasoning by identifying "important" pixels in an input image. However, the development and adoption of these methods are hindered by the lack of access to ground-truth model reasoning, which prevents accurate evaluation. In this work, we design a synthetic benchmarking framework, SMERF, that allows us to perform ground-truth-based evaluation while controlling the complexity of the model’s reasoning. Experimentally, SMERF reveals significant limitations in existing saliency methods and, as a result, represents a useful tool for the development of new saliency methods.} }
Endnote
%0 Conference Paper %T Sanity Simulations for Saliency Methods %A Joon Sik Kim %A Gregory Plumb %A Ameet Talwalkar %B Proceedings of the 39th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2022 %E Kamalika Chaudhuri %E Stefanie Jegelka %E Le Song %E Csaba Szepesvari %E Gang Niu %E Sivan Sabato %F pmlr-v162-kim22h %I PMLR %P 11173--11200 %U https://proceedings.mlr.press/v162/kim22h.html %V 162 %X Saliency methods are a popular class of feature attribution explanation methods that aim to capture a model’s predictive reasoning by identifying "important" pixels in an input image. However, the development and adoption of these methods are hindered by the lack of access to ground-truth model reasoning, which prevents accurate evaluation. In this work, we design a synthetic benchmarking framework, SMERF, that allows us to perform ground-truth-based evaluation while controlling the complexity of the model’s reasoning. Experimentally, SMERF reveals significant limitations in existing saliency methods and, as a result, represents a useful tool for the development of new saliency methods.
APA
Kim, J.S., Plumb, G. & Talwalkar, A.. (2022). Sanity Simulations for Saliency Methods. Proceedings of the 39th International Conference on Machine Learning, in Proceedings of Machine Learning Research 162:11173-11200 Available from https://proceedings.mlr.press/v162/kim22h.html.

Related Material