Human-Aligned Image Models Improve Visual Decoding from the Brain

Nona Rajabi, Antonio H. Ribeiro, Miguel Vasco, Farzaneh Taleb, Mårten Björkman, Danica Kragic
Proceedings of the 42nd International Conference on Machine Learning, PMLR 267:51009-51038, 2025.

Abstract

Decoding visual images from brain activity has significant potential for advancing brain-computer interaction and enhancing the understanding of human perception. Recent approaches align the representation spaces of images and brain activity to enable visual decoding. In this paper, we introduce the use of human-aligned image encoders to map brain signals to images. We hypothesize that these models more effectively capture perceptual attributes associated with the rapid visual stimuli presentations commonly used in visual brain data recording experiments. Our empirical results support this hypothesis, demonstrating that this simple modification improves image retrieval accuracy by up to 21% compared to state-of-the-art methods. Comprehensive experiments confirm consistent performance improvements across diverse EEG architectures, image encoders, alignment methods, participants, and brain imaging modalities.

Cite this Paper

BibTeX
@InProceedings{pmlr-v267-rajabi25a, title = {Human-Aligned Image Models Improve Visual Decoding from the Brain}, author = {Rajabi, Nona and Ribeiro, Antonio H. and Vasco, Miguel and Taleb, Farzaneh and Bj\"{o}rkman, M{\aa}rten and Kragic, Danica}, booktitle = {Proceedings of the 42nd International Conference on Machine Learning}, pages = {51009--51038}, year = {2025}, editor = {Singh, Aarti and Fazel, Maryam and Hsu, Daniel and Lacoste-Julien, Simon and Berkenkamp, Felix and Maharaj, Tegan and Wagstaff, Kiri and Zhu, Jerry}, volume = {267}, series = {Proceedings of Machine Learning Research}, month = {13--19 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v267/main/assets/rajabi25a/rajabi25a.pdf}, url = {https://proceedings.mlr.press/v267/rajabi25a.html}, abstract = {Decoding visual images from brain activity has significant potential for advancing brain-computer interaction and enhancing the understanding of human perception. Recent approaches align the representation spaces of images and brain activity to enable visual decoding. In this paper, we introduce the use of human-aligned image encoders to map brain signals to images. We hypothesize that these models more effectively capture perceptual attributes associated with the rapid visual stimuli presentations commonly used in visual brain data recording experiments. Our empirical results support this hypothesis, demonstrating that this simple modification improves image retrieval accuracy by up to 21% compared to state-of-the-art methods. Comprehensive experiments confirm consistent performance improvements across diverse EEG architectures, image encoders, alignment methods, participants, and brain imaging modalities.} }
Endnote
%0 Conference Paper %T Human-Aligned Image Models Improve Visual Decoding from the Brain %A Nona Rajabi %A Antonio H. Ribeiro %A Miguel Vasco %A Farzaneh Taleb %A Mårten Björkman %A Danica Kragic %B Proceedings of the 42nd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2025 %E Aarti Singh %E Maryam Fazel %E Daniel Hsu %E Simon Lacoste-Julien %E Felix Berkenkamp %E Tegan Maharaj %E Kiri Wagstaff %E Jerry Zhu %F pmlr-v267-rajabi25a %I PMLR %P 51009--51038 %U https://proceedings.mlr.press/v267/rajabi25a.html %V 267 %X Decoding visual images from brain activity has significant potential for advancing brain-computer interaction and enhancing the understanding of human perception. Recent approaches align the representation spaces of images and brain activity to enable visual decoding. In this paper, we introduce the use of human-aligned image encoders to map brain signals to images. We hypothesize that these models more effectively capture perceptual attributes associated with the rapid visual stimuli presentations commonly used in visual brain data recording experiments. Our empirical results support this hypothesis, demonstrating that this simple modification improves image retrieval accuracy by up to 21% compared to state-of-the-art methods. Comprehensive experiments confirm consistent performance improvements across diverse EEG architectures, image encoders, alignment methods, participants, and brain imaging modalities.
APA
Rajabi, N., Ribeiro, A.H., Vasco, M., Taleb, F., Björkman, M. & Kragic, D.. (2025). Human-Aligned Image Models Improve Visual Decoding from the Brain. Proceedings of the 42nd International Conference on Machine Learning, in Proceedings of Machine Learning Research 267:51009-51038 Available from https://proceedings.mlr.press/v267/rajabi25a.html.

Related Material