Selection of XAI Methods Matters: Evaluation of Feature Attribution Methods for Oculomotoric Biometric Identification

Daniel Krakowczyk, David Robert Reich, Paul Prasse, Sebastian Lapuschkin, Lena Ann Jäger, Tobias Scheffer
Proceedings of The 1st Gaze Meets ML workshop, PMLR 210:66-97, 2023.

Abstract

Substantial advances in oculomotoric biometric identification have been made due to deep neural networks processing non-aggregated time series data that replace methods processing theoretically motivated engineered features. However, interpretability of deep neural networks is not trivial and needs to be thoroughly investigated for future eye tracking applications. Especially in medical or legal applications explanations can be required to be provided alongside predictions. In this work, we apply several attribution methods to a state of the art model for eye movement-based biometric identification. To asses the quality of the generated attributions, this work is focused on the quantitative evaluation of a range of established metrics. We find that Layer-wise Relevance Propagation generates the least complex attributions, while DeepLIFT attributions are the most faithful. Due to the absence of a correlation between attributions of these two methods we advocate to consider both methods for their potentially complementary attributions.

Cite this Paper


BibTeX
@InProceedings{pmlr-v210-krakowczyk23a, title = {Selection of XAI Methods Matters: Evaluation of Feature Attribution Methods for Oculomotoric Biometric Identification}, author = {Krakowczyk, Daniel and Reich, David Robert and Prasse, Paul and Lapuschkin, Sebastian and J{\"a}ger, Lena Ann and Scheffer, Tobias}, booktitle = {Proceedings of The 1st Gaze Meets ML workshop}, pages = {66--97}, year = {2023}, editor = {Lourentzou, Ismini and Wu, Joy and Kashyap, Satyananda and Karargyris, Alexandros and Celi, Leo Anthony and Kawas, Ban and Talathi, Sachin}, volume = {210}, series = {Proceedings of Machine Learning Research}, month = {03 Dec}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v210/krakowczyk23a/krakowczyk23a.pdf}, url = {https://proceedings.mlr.press/v210/krakowczyk23a.html}, abstract = {Substantial advances in oculomotoric biometric identification have been made due to deep neural networks processing non-aggregated time series data that replace methods processing theoretically motivated engineered features. However, interpretability of deep neural networks is not trivial and needs to be thoroughly investigated for future eye tracking applications. Especially in medical or legal applications explanations can be required to be provided alongside predictions. In this work, we apply several attribution methods to a state of the art model for eye movement-based biometric identification. To asses the quality of the generated attributions, this work is focused on the quantitative evaluation of a range of established metrics. We find that Layer-wise Relevance Propagation generates the least complex attributions, while DeepLIFT attributions are the most faithful. Due to the absence of a correlation between attributions of these two methods we advocate to consider both methods for their potentially complementary attributions.} }
Endnote
%0 Conference Paper %T Selection of XAI Methods Matters: Evaluation of Feature Attribution Methods for Oculomotoric Biometric Identification %A Daniel Krakowczyk %A David Robert Reich %A Paul Prasse %A Sebastian Lapuschkin %A Lena Ann Jäger %A Tobias Scheffer %B Proceedings of The 1st Gaze Meets ML workshop %C Proceedings of Machine Learning Research %D 2023 %E Ismini Lourentzou %E Joy Wu %E Satyananda Kashyap %E Alexandros Karargyris %E Leo Anthony Celi %E Ban Kawas %E Sachin Talathi %F pmlr-v210-krakowczyk23a %I PMLR %P 66--97 %U https://proceedings.mlr.press/v210/krakowczyk23a.html %V 210 %X Substantial advances in oculomotoric biometric identification have been made due to deep neural networks processing non-aggregated time series data that replace methods processing theoretically motivated engineered features. However, interpretability of deep neural networks is not trivial and needs to be thoroughly investigated for future eye tracking applications. Especially in medical or legal applications explanations can be required to be provided alongside predictions. In this work, we apply several attribution methods to a state of the art model for eye movement-based biometric identification. To asses the quality of the generated attributions, this work is focused on the quantitative evaluation of a range of established metrics. We find that Layer-wise Relevance Propagation generates the least complex attributions, while DeepLIFT attributions are the most faithful. Due to the absence of a correlation between attributions of these two methods we advocate to consider both methods for their potentially complementary attributions.
APA
Krakowczyk, D., Reich, D.R., Prasse, P., Lapuschkin, S., Jäger, L.A. & Scheffer, T.. (2023). Selection of XAI Methods Matters: Evaluation of Feature Attribution Methods for Oculomotoric Biometric Identification. Proceedings of The 1st Gaze Meets ML workshop, in Proceedings of Machine Learning Research 210:66-97 Available from https://proceedings.mlr.press/v210/krakowczyk23a.html.

Related Material