Pre-Training Transformers for Fingerprinting to Improve Stress Prediction in fMRI

Gony Rosenman, Itzik Malkiel, Ayam Greental, Talma Hendler, Lior Wolf
Medical Imaging with Deep Learning, PMLR 227:212-234, 2024.

Abstract

We harness a Transformer-based model and a pre-training procedure for fingerprinting on fMRI data, to enhance the accuracy of stress predictions. Our model, called MetricFMRI, first optimizes a pixel-based reconstruction loss. In a second unsupervised training phase, a triplet loss is used to encourage fMRI sequences of the same subject to have closer representations, while sequences from different subjects are pushed away from each other. Finally, supervised learning is used for the target task, based on the learned representation. We evaluate the performance of our model and other alternatives and conclude that the triplet training for the fingerprinting task is key to the improved accuracy of our method for the task of stress prediction. To obtain insights regarding the learned model, gradient-based explainability techniques are used, indicating that sub-cortical brain regions that are known to play a central role in stress-related processes are highlighted by the model.

Cite this Paper

BibTeX
@InProceedings{pmlr-v227-rosenman24a, title = {Pre-Training Transformers for Fingerprinting to Improve Stress Prediction in fMRI}, author = {Rosenman, Gony and Malkiel, Itzik and Greental, Ayam and Hendler, Talma and Wolf, Lior}, booktitle = {Medical Imaging with Deep Learning}, pages = {212--234}, year = {2024}, editor = {Oguz, Ipek and Noble, Jack and Li, Xiaoxiao and Styner, Martin and Baumgartner, Christian and Rusu, Mirabela and Heinmann, Tobias and Kontos, Despina and Landman, Bennett and Dawant, Benoit}, volume = {227}, series = {Proceedings of Machine Learning Research}, month = {10--12 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v227/rosenman24a/rosenman24a.pdf}, url = {https://proceedings.mlr.press/v227/rosenman24a.html}, abstract = {We harness a Transformer-based model and a pre-training procedure for fingerprinting on fMRI data, to enhance the accuracy of stress predictions. Our model, called MetricFMRI, first optimizes a pixel-based reconstruction loss. In a second unsupervised training phase, a triplet loss is used to encourage fMRI sequences of the same subject to have closer representations, while sequences from different subjects are pushed away from each other. Finally, supervised learning is used for the target task, based on the learned representation. We evaluate the performance of our model and other alternatives and conclude that the triplet training for the fingerprinting task is key to the improved accuracy of our method for the task of stress prediction. To obtain insights regarding the learned model, gradient-based explainability techniques are used, indicating that sub-cortical brain regions that are known to play a central role in stress-related processes are highlighted by the model.} }
Endnote
%0 Conference Paper %T Pre-Training Transformers for Fingerprinting to Improve Stress Prediction in fMRI %A Gony Rosenman %A Itzik Malkiel %A Ayam Greental %A Talma Hendler %A Lior Wolf %B Medical Imaging with Deep Learning %C Proceedings of Machine Learning Research %D 2024 %E Ipek Oguz %E Jack Noble %E Xiaoxiao Li %E Martin Styner %E Christian Baumgartner %E Mirabela Rusu %E Tobias Heinmann %E Despina Kontos %E Bennett Landman %E Benoit Dawant %F pmlr-v227-rosenman24a %I PMLR %P 212--234 %U https://proceedings.mlr.press/v227/rosenman24a.html %V 227 %X We harness a Transformer-based model and a pre-training procedure for fingerprinting on fMRI data, to enhance the accuracy of stress predictions. Our model, called MetricFMRI, first optimizes a pixel-based reconstruction loss. In a second unsupervised training phase, a triplet loss is used to encourage fMRI sequences of the same subject to have closer representations, while sequences from different subjects are pushed away from each other. Finally, supervised learning is used for the target task, based on the learned representation. We evaluate the performance of our model and other alternatives and conclude that the triplet training for the fingerprinting task is key to the improved accuracy of our method for the task of stress prediction. To obtain insights regarding the learned model, gradient-based explainability techniques are used, indicating that sub-cortical brain regions that are known to play a central role in stress-related processes are highlighted by the model.
APA
Rosenman, G., Malkiel, I., Greental, A., Hendler, T. & Wolf, L.. (2024). Pre-Training Transformers for Fingerprinting to Improve Stress Prediction in fMRI. Medical Imaging with Deep Learning, in Proceedings of Machine Learning Research 227:212-234 Available from https://proceedings.mlr.press/v227/rosenman24a.html.

Related Material