ROAMM: A Benchmark Dataset for Multimodal Human Attention Decoding and EEG-to-Text Modeling During Naturalistic Reading

Haorui Sun, Ardyn Vivienne Olszko, Niharika Singh, David C. Jangraw
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:117082-117099, 2026.

Abstract

We present Reading Observed At Mindless Moments (ROAMM), a multimodal dataset comprising 50 hours of simultaneous EEG and eye-tracking recordings collected during naturalistic multi-page reading from 44 participants. ROAMM includes synchronized physiological recordings, eye-movement events, page-level comprehension scores, and span-level mind-wandering (MW) annotations obtained using a retrospective self-report paradigm. We introduce a standardized leave-one-subject-out benchmark for MW detection and achieve up to 0.609 AUROC using supervised models. We additionally evaluate EEG-to-text decoding on reading segments with and without MW labels, showing that decoding performance decreases during MW episodes. ROAMM enables research on MW detection, EEG-to-text decoding, multimodal representation learning, and attention-related degradation of language representations during naturalistic reading.

Cite this Paper


BibTeX
@InProceedings{pmlr-v306-sun26ag, title = {{ROAMM}: A Benchmark Dataset for Multimodal Human Attention Decoding and {EEG}-to-Text Modeling During Naturalistic Reading}, author = {Sun, Haorui and Olszko, Ardyn Vivienne and Singh, Niharika and Jangraw, David C.}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {117082--117099}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/sun26ag/sun26ag.pdf}, url = {https://proceedings.mlr.press/v306/sun26ag.html}, abstract = {We present Reading Observed At Mindless Moments (ROAMM), a multimodal dataset comprising 50 hours of simultaneous EEG and eye-tracking recordings collected during naturalistic multi-page reading from 44 participants. ROAMM includes synchronized physiological recordings, eye-movement events, page-level comprehension scores, and span-level mind-wandering (MW) annotations obtained using a retrospective self-report paradigm. We introduce a standardized leave-one-subject-out benchmark for MW detection and achieve up to 0.609 AUROC using supervised models. We additionally evaluate EEG-to-text decoding on reading segments with and without MW labels, showing that decoding performance decreases during MW episodes. ROAMM enables research on MW detection, EEG-to-text decoding, multimodal representation learning, and attention-related degradation of language representations during naturalistic reading.} }
Endnote
%0 Conference Paper %T ROAMM: A Benchmark Dataset for Multimodal Human Attention Decoding and EEG-to-Text Modeling During Naturalistic Reading %A Haorui Sun %A Ardyn Vivienne Olszko %A Niharika Singh %A David C. Jangraw %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-sun26ag %I PMLR %P 117082--117099 %U https://proceedings.mlr.press/v306/sun26ag.html %V 306 %X We present Reading Observed At Mindless Moments (ROAMM), a multimodal dataset comprising 50 hours of simultaneous EEG and eye-tracking recordings collected during naturalistic multi-page reading from 44 participants. ROAMM includes synchronized physiological recordings, eye-movement events, page-level comprehension scores, and span-level mind-wandering (MW) annotations obtained using a retrospective self-report paradigm. We introduce a standardized leave-one-subject-out benchmark for MW detection and achieve up to 0.609 AUROC using supervised models. We additionally evaluate EEG-to-text decoding on reading segments with and without MW labels, showing that decoding performance decreases during MW episodes. ROAMM enables research on MW detection, EEG-to-text decoding, multimodal representation learning, and attention-related degradation of language representations during naturalistic reading.
APA
Sun, H., Olszko, A.V., Singh, N. & Jangraw, D.C.. (2026). ROAMM: A Benchmark Dataset for Multimodal Human Attention Decoding and EEG-to-Text Modeling During Naturalistic Reading. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:117082-117099 Available from https://proceedings.mlr.press/v306/sun26ag.html.

Related Material