What does LIME really see in images?

Damien Garreau, Dina Mardaoui
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:3620-3629, 2021.

Abstract

The performance of modern algorithms on certain computer vision tasks such as object recognition is now close to that of humans. This success was achieved at the price of complicated architectures depending on millions of parameters and it has become quite challenging to understand how particular predictions are made. Interpretability methods propose to give us this understanding. In this paper, we study LIME, perhaps one of the most popular. On the theoretical side, we show that when the number of generated examples is large, LIME explanations are concentrated around a limit explanation for which we give an explicit expression. We further this study for elementary shape detectors and linear models. As a consequence of this analysis, we uncover a connection between LIME and integrated gradients, another explanation method. More precisely, the LIME explanations are similar to the sum of integrated gradients over the superpixels used in the preprocessing step of LIME.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-garreau21a, title = {What does LIME really see in images?}, author = {Garreau, Damien and Mardaoui, Dina}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {3620--3629}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/garreau21a/garreau21a.pdf}, url = {https://proceedings.mlr.press/v139/garreau21a.html}, abstract = {The performance of modern algorithms on certain computer vision tasks such as object recognition is now close to that of humans. This success was achieved at the price of complicated architectures depending on millions of parameters and it has become quite challenging to understand how particular predictions are made. Interpretability methods propose to give us this understanding. In this paper, we study LIME, perhaps one of the most popular. On the theoretical side, we show that when the number of generated examples is large, LIME explanations are concentrated around a limit explanation for which we give an explicit expression. We further this study for elementary shape detectors and linear models. As a consequence of this analysis, we uncover a connection between LIME and integrated gradients, another explanation method. More precisely, the LIME explanations are similar to the sum of integrated gradients over the superpixels used in the preprocessing step of LIME.} }
Endnote
%0 Conference Paper %T What does LIME really see in images? %A Damien Garreau %A Dina Mardaoui %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-garreau21a %I PMLR %P 3620--3629 %U https://proceedings.mlr.press/v139/garreau21a.html %V 139 %X The performance of modern algorithms on certain computer vision tasks such as object recognition is now close to that of humans. This success was achieved at the price of complicated architectures depending on millions of parameters and it has become quite challenging to understand how particular predictions are made. Interpretability methods propose to give us this understanding. In this paper, we study LIME, perhaps one of the most popular. On the theoretical side, we show that when the number of generated examples is large, LIME explanations are concentrated around a limit explanation for which we give an explicit expression. We further this study for elementary shape detectors and linear models. As a consequence of this analysis, we uncover a connection between LIME and integrated gradients, another explanation method. More precisely, the LIME explanations are similar to the sum of integrated gradients over the superpixels used in the preprocessing step of LIME.
APA
Garreau, D. & Mardaoui, D.. (2021). What does LIME really see in images?. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:3620-3629 Available from https://proceedings.mlr.press/v139/garreau21a.html.

Related Material