FlexR: Few-shot Classification with Language Embeddings for Structured Reporting of Chest X-rays

Matthias Keicher, Kamilia Zaripova, Tobias Czempiel, Kristina Mach, Ashkan Khakzar, Nassir Navab
Medical Imaging with Deep Learning, PMLR 227:1493-1508, 2024.

Abstract

The automation of chest X-ray reporting has garnered significant interest due to the time-consuming nature of the task. However, the clinical accuracy of free-text reports has proven challenging to quantify using natural language processing metrics, given the complexity of medical information, the variety of writing styles, and the potential for typos and inconsistencies. Structured reporting and standardized reports, on the other hand, can provide consistency and formalize the evaluation of clinical correctness. However, high-quality annotations for structured reporting are scarce. Therefore, we propose a method to predict clinical findings defined by sentences in structured reporting templates, which can be used to fill such templates. The approach involves training a contrastive language-image model using chest X-rays and related free-text radiological reports, then creating textual prompts for each structured finding and optimizing a classifier to predict clinical findings in the medical image. Results show that even with limited image-level annotations for training, the method can accomplish the structured reporting tasks of severity assessment of cardiomegaly and localizing pathologies in chest X-rays.

Cite this Paper

BibTeX
@InProceedings{pmlr-v227-keicher24a, title = {FlexR: Few-shot Classification with Language Embeddings for Structured Reporting of Chest X-rays}, author = {Keicher, Matthias and Zaripova, Kamilia and Czempiel, Tobias and Mach, Kristina and Khakzar, Ashkan and Navab, Nassir}, booktitle = {Medical Imaging with Deep Learning}, pages = {1493--1508}, year = {2024}, editor = {Oguz, Ipek and Noble, Jack and Li, Xiaoxiao and Styner, Martin and Baumgartner, Christian and Rusu, Mirabela and Heinmann, Tobias and Kontos, Despina and Landman, Bennett and Dawant, Benoit}, volume = {227}, series = {Proceedings of Machine Learning Research}, month = {10--12 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v227/keicher24a/keicher24a.pdf}, url = {https://proceedings.mlr.press/v227/keicher24a.html}, abstract = {The automation of chest X-ray reporting has garnered significant interest due to the time-consuming nature of the task. However, the clinical accuracy of free-text reports has proven challenging to quantify using natural language processing metrics, given the complexity of medical information, the variety of writing styles, and the potential for typos and inconsistencies. Structured reporting and standardized reports, on the other hand, can provide consistency and formalize the evaluation of clinical correctness. However, high-quality annotations for structured reporting are scarce. Therefore, we propose a method to predict clinical findings defined by sentences in structured reporting templates, which can be used to fill such templates. The approach involves training a contrastive language-image model using chest X-rays and related free-text radiological reports, then creating textual prompts for each structured finding and optimizing a classifier to predict clinical findings in the medical image. Results show that even with limited image-level annotations for training, the method can accomplish the structured reporting tasks of severity assessment of cardiomegaly and localizing pathologies in chest X-rays.} }
Endnote
%0 Conference Paper %T FlexR: Few-shot Classification with Language Embeddings for Structured Reporting of Chest X-rays %A Matthias Keicher %A Kamilia Zaripova %A Tobias Czempiel %A Kristina Mach %A Ashkan Khakzar %A Nassir Navab %B Medical Imaging with Deep Learning %C Proceedings of Machine Learning Research %D 2024 %E Ipek Oguz %E Jack Noble %E Xiaoxiao Li %E Martin Styner %E Christian Baumgartner %E Mirabela Rusu %E Tobias Heinmann %E Despina Kontos %E Bennett Landman %E Benoit Dawant %F pmlr-v227-keicher24a %I PMLR %P 1493--1508 %U https://proceedings.mlr.press/v227/keicher24a.html %V 227 %X The automation of chest X-ray reporting has garnered significant interest due to the time-consuming nature of the task. However, the clinical accuracy of free-text reports has proven challenging to quantify using natural language processing metrics, given the complexity of medical information, the variety of writing styles, and the potential for typos and inconsistencies. Structured reporting and standardized reports, on the other hand, can provide consistency and formalize the evaluation of clinical correctness. However, high-quality annotations for structured reporting are scarce. Therefore, we propose a method to predict clinical findings defined by sentences in structured reporting templates, which can be used to fill such templates. The approach involves training a contrastive language-image model using chest X-rays and related free-text radiological reports, then creating textual prompts for each structured finding and optimizing a classifier to predict clinical findings in the medical image. Results show that even with limited image-level annotations for training, the method can accomplish the structured reporting tasks of severity assessment of cardiomegaly and localizing pathologies in chest X-rays.
APA
Keicher, M., Zaripova, K., Czempiel, T., Mach, K., Khakzar, A. & Navab, N.. (2024). FlexR: Few-shot Classification with Language Embeddings for Structured Reporting of Chest X-rays. Medical Imaging with Deep Learning, in Proceedings of Machine Learning Research 227:1493-1508 Available from https://proceedings.mlr.press/v227/keicher24a.html.

Related Material