Learning to Automatically Generate Accurate ECG Captions

Mathieu G. G. Bartels, Ivona Najdenkoska, Rutger R van de Leur, Arjan Sammani, Karim Taha, David M Knigge, Pieter A Doevendans, Marcel Worring, René van Es
Proceedings of The 5th International Conference on Medical Imaging with Deep Learning, PMLR 172:86-102, 2022.

Abstract

The electrocardiogram (ECG) is an affordable, non-invasive and quick method to gain essential information about the electrical activity of the heart. Interpreting ECGs is a time-consuming process even for experienced cardiologists, which motivates the current usage of rule-based methods in clinical practice to automatically describe ECGs. However, in comparison with descriptions created by experts, ECG-descriptions generated by such rule-based methods show considerable limitations. Inspired by image captioning methods, we instead propose a data-driven approach for ECG description generation. We introduce a label-guided Transformer model, and show that it is possible to automatically generate relevant and readable ECG descriptions with a data-driven captioning model. We incorporate prior ECG labels into our model design, and show this improves the overall quality of generated descriptions. We find that training these models on free-text annotations of ECGs - instead of the clinically-used computer generated ECG descriptions - greatly improves performance. Moreover, we perform a human expert evaluation study of our best system, which shows that our data-driven approach improves upon existing rule-based methods.

Cite this Paper

BibTeX
@InProceedings{pmlr-v172-bartels22a, title = {Learning to Automatically Generate Accurate ECG Captions}, author = {Bartels, Mathieu G. G. and Najdenkoska, Ivona and van de Leur, Rutger R and Sammani, Arjan and Taha, Karim and Knigge, David M and Doevendans, Pieter A and Worring, Marcel and van Es, Ren{\'e}}, booktitle = {Proceedings of The 5th International Conference on Medical Imaging with Deep Learning}, pages = {86--102}, year = {2022}, editor = {Konukoglu, Ender and Menze, Bjoern and Venkataraman, Archana and Baumgartner, Christian and Dou, Qi and Albarqouni, Shadi}, volume = {172}, series = {Proceedings of Machine Learning Research}, month = {06--08 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v172/bartels22a/bartels22a.pdf}, url = {https://proceedings.mlr.press/v172/bartels22a.html}, abstract = {The electrocardiogram (ECG) is an affordable, non-invasive and quick method to gain essential information about the electrical activity of the heart. Interpreting ECGs is a time-consuming process even for experienced cardiologists, which motivates the current usage of rule-based methods in clinical practice to automatically describe ECGs. However, in comparison with descriptions created by experts, ECG-descriptions generated by such rule-based methods show considerable limitations. Inspired by image captioning methods, we instead propose a data-driven approach for ECG description generation. We introduce a label-guided Transformer model, and show that it is possible to automatically generate relevant and readable ECG descriptions with a data-driven captioning model. We incorporate prior ECG labels into our model design, and show this improves the overall quality of generated descriptions. We find that training these models on free-text annotations of ECGs - instead of the clinically-used computer generated ECG descriptions - greatly improves performance. Moreover, we perform a human expert evaluation study of our best system, which shows that our data-driven approach improves upon existing rule-based methods.} }
Endnote
%0 Conference Paper %T Learning to Automatically Generate Accurate ECG Captions %A Mathieu G. G. Bartels %A Ivona Najdenkoska %A Rutger R van de Leur %A Arjan Sammani %A Karim Taha %A David M Knigge %A Pieter A Doevendans %A Marcel Worring %A René van Es %B Proceedings of The 5th International Conference on Medical Imaging with Deep Learning %C Proceedings of Machine Learning Research %D 2022 %E Ender Konukoglu %E Bjoern Menze %E Archana Venkataraman %E Christian Baumgartner %E Qi Dou %E Shadi Albarqouni %F pmlr-v172-bartels22a %I PMLR %P 86--102 %U https://proceedings.mlr.press/v172/bartels22a.html %V 172 %X The electrocardiogram (ECG) is an affordable, non-invasive and quick method to gain essential information about the electrical activity of the heart. Interpreting ECGs is a time-consuming process even for experienced cardiologists, which motivates the current usage of rule-based methods in clinical practice to automatically describe ECGs. However, in comparison with descriptions created by experts, ECG-descriptions generated by such rule-based methods show considerable limitations. Inspired by image captioning methods, we instead propose a data-driven approach for ECG description generation. We introduce a label-guided Transformer model, and show that it is possible to automatically generate relevant and readable ECG descriptions with a data-driven captioning model. We incorporate prior ECG labels into our model design, and show this improves the overall quality of generated descriptions. We find that training these models on free-text annotations of ECGs - instead of the clinically-used computer generated ECG descriptions - greatly improves performance. Moreover, we perform a human expert evaluation study of our best system, which shows that our data-driven approach improves upon existing rule-based methods.
APA
Bartels, M.G.G., Najdenkoska, I., van de Leur, R.R., Sammani, A., Taha, K., Knigge, D.M., Doevendans, P.A., Worring, M. & van Es, R.. (2022). Learning to Automatically Generate Accurate ECG Captions. Proceedings of The 5th International Conference on Medical Imaging with Deep Learning, in Proceedings of Machine Learning Research 172:86-102 Available from https://proceedings.mlr.press/v172/bartels22a.html.

Related Material