Generating Attention Maps from Eye-gaze for the Diagnosis of Alzheimer’s Disease

Carlos Antunes, Margarida Silveira
Proceedings of The 1st Gaze Meets ML workshop, PMLR 210:3-19, 2023.

Abstract

Convolutional neural networks (CNNs) are currently the best computational methods for the diagnosis of Alzheimer’s disease (AD) from neuroimaging. CNNs are able to automati- cally learn a hierarchy of spatial features, but they are not optimized to incorporate domain knowledge. In this work we study the generation of attention maps based on a human expert gaze of the brain scans (domain knowledge) to guide the deep model to focus on the more relevant regions for AD diagnosis. Two strategies to generate the maps from eye-gaze were investigated; the use of average class maps and supervising a network to generate the attention maps. These approaches were compared with masking (hard attention) with regions of interest (ROI) and CNNs with traditional attention mechanisms. For our experiments, we used positron emission tomography (PET) scans from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database. For the task of normal control (NC) vs Alzheimer’s (AD), the best performing model was with insertion of regions of interest (ROI), which achieved 95.6% accuracy, 0.4% higher than the baseline CNN. Keywords: Deep learning; Alzheimer’s disease; Convolutional neural network; Attention mechanism; Eye tracking; Computer-aided diagnosis.

Cite this Paper

BibTeX
@InProceedings{pmlr-v210-antunes23a, title = {Generating Attention Maps from Eye-gaze for the Diagnosis of Alzheimer’s Disease}, author = {Antunes, Carlos and Silveira, Margarida}, booktitle = {Proceedings of The 1st Gaze Meets ML workshop}, pages = {3--19}, year = {2023}, editor = {Lourentzou, Ismini and Wu, Joy and Kashyap, Satyananda and Karargyris, Alexandros and Celi, Leo Anthony and Kawas, Ban and Talathi, Sachin}, volume = {210}, series = {Proceedings of Machine Learning Research}, month = {03 Dec}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v210/antunes23a/antunes23a.pdf}, url = {https://proceedings.mlr.press/v210/antunes23a.html}, abstract = {Convolutional neural networks (CNNs) are currently the best computational methods for the diagnosis of Alzheimer’s disease (AD) from neuroimaging. CNNs are able to automati- cally learn a hierarchy of spatial features, but they are not optimized to incorporate domain knowledge. In this work we study the generation of attention maps based on a human expert gaze of the brain scans (domain knowledge) to guide the deep model to focus on the more relevant regions for AD diagnosis. Two strategies to generate the maps from eye-gaze were investigated; the use of average class maps and supervising a network to generate the attention maps. These approaches were compared with masking (hard attention) with regions of interest (ROI) and CNNs with traditional attention mechanisms. For our experiments, we used positron emission tomography (PET) scans from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database. For the task of normal control (NC) vs Alzheimer’s (AD), the best performing model was with insertion of regions of interest (ROI), which achieved 95.6% accuracy, 0.4% higher than the baseline CNN. Keywords: Deep learning; Alzheimer’s disease; Convolutional neural network; Attention mechanism; Eye tracking; Computer-aided diagnosis.} }
Endnote
%0 Conference Paper %T Generating Attention Maps from Eye-gaze for the Diagnosis of Alzheimer’s Disease %A Carlos Antunes %A Margarida Silveira %B Proceedings of The 1st Gaze Meets ML workshop %C Proceedings of Machine Learning Research %D 2023 %E Ismini Lourentzou %E Joy Wu %E Satyananda Kashyap %E Alexandros Karargyris %E Leo Anthony Celi %E Ban Kawas %E Sachin Talathi %F pmlr-v210-antunes23a %I PMLR %P 3--19 %U https://proceedings.mlr.press/v210/antunes23a.html %V 210 %X Convolutional neural networks (CNNs) are currently the best computational methods for the diagnosis of Alzheimer’s disease (AD) from neuroimaging. CNNs are able to automati- cally learn a hierarchy of spatial features, but they are not optimized to incorporate domain knowledge. In this work we study the generation of attention maps based on a human expert gaze of the brain scans (domain knowledge) to guide the deep model to focus on the more relevant regions for AD diagnosis. Two strategies to generate the maps from eye-gaze were investigated; the use of average class maps and supervising a network to generate the attention maps. These approaches were compared with masking (hard attention) with regions of interest (ROI) and CNNs with traditional attention mechanisms. For our experiments, we used positron emission tomography (PET) scans from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database. For the task of normal control (NC) vs Alzheimer’s (AD), the best performing model was with insertion of regions of interest (ROI), which achieved 95.6% accuracy, 0.4% higher than the baseline CNN. Keywords: Deep learning; Alzheimer’s disease; Convolutional neural network; Attention mechanism; Eye tracking; Computer-aided diagnosis.
APA
Antunes, C. & Silveira, M.. (2023). Generating Attention Maps from Eye-gaze for the Diagnosis of Alzheimer’s Disease. Proceedings of The 1st Gaze Meets ML workshop, in Proceedings of Machine Learning Research 210:3-19 Available from https://proceedings.mlr.press/v210/antunes23a.html.

Related Material