[edit]
Indication Driven Autoregressive Report Generation for Cardiac Magnetic Resonance Imaging
Proceedings of the 4th Machine Learning for Health Symposium, PMLR 259:775-786, 2025.
Abstract
Interpreting and documenting findings from cardiac imaging studies is increasingly burdensome to readers in part due to the increasing amount of advanced cardiac imaging studies which capture multi-parametric data. This is particularly true of cardiac magnetic resonance imaging (CMR) studies which encode features of morphology, function, flow, parametric mapping, and myocardial viability in multiple 2D planes, but require a substantial amount of time to analyze, document, and integrate the numerous complex imaging features into a comprehensive report. Additionally, clearly communicating complex CMR findings and diagnoses to referring physicians with varying CMR knowledge and the ability to clinically correlated complex CMR findings is highly variable. Automatic interpretation and generation of the report have great potential to reduce the burden on readers and improve access through higher patient throughput. As such, there has been significant work in this area, although much of it has been focused on more simplistic chest X-ray and single view echocardiography. These data sources are represented by only a single view or have only a single source of contrast, greatly reducing the necessary complexity of the latent visual space. Furthermore, we recognize that clinical histories are important for accurate reporting. In this work, we propose to treat the CMR study as a multi-scene video and generate the corresponding report in an autoregressive manner. We further warm-start the generated report with the indications for the exam to improve the relevance of the generated report. We validate our model on two closed CMR datasets from two different institutions and demonstrate that our model offers significant improvements on both language generation metrics and human reader preference.