Indication Driven Autoregressive Report Generation for Cardiac Magnetic Resonance Imaging

Makiya Nakashima, Po-Hao Chen, Michael Bolen, Christopher Nguyen, W. H. Wilson Tang, Richard Grimm, Deborah Kwon, David Chen
Proceedings of the 4th Machine Learning for Health Symposium, PMLR 259:775-786, 2025.

Abstract

Interpreting and documenting findings from cardiac imaging studies is increasingly burdensome to readers in part due to the increasing amount of advanced cardiac imaging studies which capture multi-parametric data. This is particularly true of cardiac magnetic resonance imaging (CMR) studies which encode features of morphology, function, flow, parametric mapping, and myocardial viability in multiple 2D planes, but require a substantial amount of time to analyze, document, and integrate the numerous complex imaging features into a comprehensive report. Additionally, clearly communicating complex CMR findings and diagnoses to referring physicians with varying CMR knowledge and the ability to clinically correlated complex CMR findings is highly variable. Automatic interpretation and generation of the report have great potential to reduce the burden on readers and improve access through higher patient throughput. As such, there has been significant work in this area, although much of it has been focused on more simplistic chest X-ray and single view echocardiography. These data sources are represented by only a single view or have only a single source of contrast, greatly reducing the necessary complexity of the latent visual space. Furthermore, we recognize that clinical histories are important for accurate reporting. In this work, we propose to treat the CMR study as a multi-scene video and generate the corresponding report in an autoregressive manner. We further warm-start the generated report with the indications for the exam to improve the relevance of the generated report. We validate our model on two closed CMR datasets from two different institutions and demonstrate that our model offers significant improvements on both language generation metrics and human reader preference.

Cite this Paper

BibTeX
@InProceedings{pmlr-v259-nakashima25a, title = {Indication Driven Autoregressive Report Generation for Cardiac Magnetic Resonance Imaging}, author = {Nakashima, Makiya and Chen, Po-Hao and Bolen, Michael and Nguyen, Christopher and Tang, W. H. Wilson and Grimm, Richard and Kwon, Deborah and Chen, David}, booktitle = {Proceedings of the 4th Machine Learning for Health Symposium}, pages = {775--786}, year = {2025}, editor = {Hegselmann, Stefan and Zhou, Helen and Healey, Elizabeth and Chang, Trenton and Ellington, Caleb and Mhasawade, Vishwali and Tonekaboni, Sana and Argaw, Peniel and Zhang, Haoran}, volume = {259}, series = {Proceedings of Machine Learning Research}, month = {15--16 Dec}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v259/main/assets/nakashima25a/nakashima25a.pdf}, url = {https://proceedings.mlr.press/v259/nakashima25a.html}, abstract = {Interpreting and documenting findings from cardiac imaging studies is increasingly burdensome to readers in part due to the increasing amount of advanced cardiac imaging studies which capture multi-parametric data. This is particularly true of cardiac magnetic resonance imaging (CMR) studies which encode features of morphology, function, flow, parametric mapping, and myocardial viability in multiple 2D planes, but require a substantial amount of time to analyze, document, and integrate the numerous complex imaging features into a comprehensive report. Additionally, clearly communicating complex CMR findings and diagnoses to referring physicians with varying CMR knowledge and the ability to clinically correlated complex CMR findings is highly variable. Automatic interpretation and generation of the report have great potential to reduce the burden on readers and improve access through higher patient throughput. As such, there has been significant work in this area, although much of it has been focused on more simplistic chest X-ray and single view echocardiography. These data sources are represented by only a single view or have only a single source of contrast, greatly reducing the necessary complexity of the latent visual space. Furthermore, we recognize that clinical histories are important for accurate reporting. In this work, we propose to treat the CMR study as a multi-scene video and generate the corresponding report in an autoregressive manner. We further warm-start the generated report with the indications for the exam to improve the relevance of the generated report. We validate our model on two closed CMR datasets from two different institutions and demonstrate that our model offers significant improvements on both language generation metrics and human reader preference.} }
Endnote
%0 Conference Paper %T Indication Driven Autoregressive Report Generation for Cardiac Magnetic Resonance Imaging %A Makiya Nakashima %A Po-Hao Chen %A Michael Bolen %A Christopher Nguyen %A W. H. Wilson Tang %A Richard Grimm %A Deborah Kwon %A David Chen %B Proceedings of the 4th Machine Learning for Health Symposium %C Proceedings of Machine Learning Research %D 2025 %E Stefan Hegselmann %E Helen Zhou %E Elizabeth Healey %E Trenton Chang %E Caleb Ellington %E Vishwali Mhasawade %E Sana Tonekaboni %E Peniel Argaw %E Haoran Zhang %F pmlr-v259-nakashima25a %I PMLR %P 775--786 %U https://proceedings.mlr.press/v259/nakashima25a.html %V 259 %X Interpreting and documenting findings from cardiac imaging studies is increasingly burdensome to readers in part due to the increasing amount of advanced cardiac imaging studies which capture multi-parametric data. This is particularly true of cardiac magnetic resonance imaging (CMR) studies which encode features of morphology, function, flow, parametric mapping, and myocardial viability in multiple 2D planes, but require a substantial amount of time to analyze, document, and integrate the numerous complex imaging features into a comprehensive report. Additionally, clearly communicating complex CMR findings and diagnoses to referring physicians with varying CMR knowledge and the ability to clinically correlated complex CMR findings is highly variable. Automatic interpretation and generation of the report have great potential to reduce the burden on readers and improve access through higher patient throughput. As such, there has been significant work in this area, although much of it has been focused on more simplistic chest X-ray and single view echocardiography. These data sources are represented by only a single view or have only a single source of contrast, greatly reducing the necessary complexity of the latent visual space. Furthermore, we recognize that clinical histories are important for accurate reporting. In this work, we propose to treat the CMR study as a multi-scene video and generate the corresponding report in an autoregressive manner. We further warm-start the generated report with the indications for the exam to improve the relevance of the generated report. We validate our model on two closed CMR datasets from two different institutions and demonstrate that our model offers significant improvements on both language generation metrics and human reader preference.
APA
Nakashima, M., Chen, P., Bolen, M., Nguyen, C., Tang, W.H.W., Grimm, R., Kwon, D. & Chen, D.. (2025). Indication Driven Autoregressive Report Generation for Cardiac Magnetic Resonance Imaging. Proceedings of the 4th Machine Learning for Health Symposium, in Proceedings of Machine Learning Research 259:775-786 Available from https://proceedings.mlr.press/v259/nakashima25a.html.

Related Material