A Deep Learning Based Framework for Joint Image Registration and Segmentation of Brain Metastases on Magnetic Resonance Imaging

Jay Patel, Syed Rakin Ahmed, Ken Chang, Praveer Singh, Mishka Gidwani, Katharina Hoebel, Albert Kim, Christopher Bridge, Chung-Jen Teng, Xiaomei Li, Gongwen Xu, Megan McDonald, Ayal Aizer, Wenya Linda Bi, Ina Ly, Bruce Rosen, Priscilla Brastianos, Raymond Huang, Elizabeth Gerstner, Jayashree Kalpathy-Cramer
Proceedings of the 8th Machine Learning for Healthcare Conference, PMLR 219:565-587, 2023.

Abstract

Manual segmentation of brain metastases (BM) is a laborious and time-consuming task for expert clinicians, especially in the setting of longitudinal patient imaging. Although automated deep learning (DL) approaches can segment larger lesions effectively, they suffer from poor sensitivity of lesion detection for micro-metastases. Moreover, these approaches segment all patient imaging independently of each other, ignoring relevant information from prior time-points. In order to utilize prior time-point information, we propose SPIRS, a joint image registration and segmentation method. Given a prior time-point image and segmentation mask (which are readily available in a routine clinical environment), we affinely and deformably register these to a new time-point image. This warped prior image and mask are then used to enhance and improve the segmentation of the new time-point. We apply SPIRS to a large retrospectively acquired single institution dataset and show that it outperforms current registration approaches on BM imaging and that it significantly improves segmentation performance for micro-metastatic lesions.

Cite this Paper

BibTeX
@InProceedings{pmlr-v219-patel23a, title = {A Deep Learning Based Framework for Joint Image Registration and Segmentation of Brain Metastases on Magnetic Resonance Imaging}, author = {Patel, Jay and Ahmed, Syed Rakin and Chang, Ken and Singh, Praveer and Gidwani, Mishka and Hoebel, Katharina and Kim, Albert and Bridge, Christopher and Teng, Chung-Jen and Li, Xiaomei and Xu, Gongwen and McDonald, Megan and Aizer, Ayal and Bi, Wenya Linda and Ly, Ina and Rosen, Bruce and Brastianos, Priscilla and Huang, Raymond and Gerstner, Elizabeth and Kalpathy-Cramer, Jayashree}, booktitle = {Proceedings of the 8th Machine Learning for Healthcare Conference}, pages = {565--587}, year = {2023}, editor = {Deshpande, Kaivalya and Fiterau, Madalina and Joshi, Shalmali and Lipton, Zachary and Ranganath, Rajesh and Urteaga, Iñigo and Yeung, Serene}, volume = {219}, series = {Proceedings of Machine Learning Research}, month = {11--12 Aug}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v219/patel23a/patel23a.pdf}, url = {https://proceedings.mlr.press/v219/patel23a.html}, abstract = {Manual segmentation of brain metastases (BM) is a laborious and time-consuming task for expert clinicians, especially in the setting of longitudinal patient imaging. Although automated deep learning (DL) approaches can segment larger lesions effectively, they suffer from poor sensitivity of lesion detection for micro-metastases. Moreover, these approaches segment all patient imaging independently of each other, ignoring relevant information from prior time-points. In order to utilize prior time-point information, we propose SPIRS, a joint image registration and segmentation method. Given a prior time-point image and segmentation mask (which are readily available in a routine clinical environment), we affinely and deformably register these to a new time-point image. This warped prior image and mask are then used to enhance and improve the segmentation of the new time-point. We apply SPIRS to a large retrospectively acquired single institution dataset and show that it outperforms current registration approaches on BM imaging and that it significantly improves segmentation performance for micro-metastatic lesions.} }
Endnote
%0 Conference Paper %T A Deep Learning Based Framework for Joint Image Registration and Segmentation of Brain Metastases on Magnetic Resonance Imaging %A Jay Patel %A Syed Rakin Ahmed %A Ken Chang %A Praveer Singh %A Mishka Gidwani %A Katharina Hoebel %A Albert Kim %A Christopher Bridge %A Chung-Jen Teng %A Xiaomei Li %A Gongwen Xu %A Megan McDonald %A Ayal Aizer %A Wenya Linda Bi %A Ina Ly %A Bruce Rosen %A Priscilla Brastianos %A Raymond Huang %A Elizabeth Gerstner %A Jayashree Kalpathy-Cramer %B Proceedings of the 8th Machine Learning for Healthcare Conference %C Proceedings of Machine Learning Research %D 2023 %E Kaivalya Deshpande %E Madalina Fiterau %E Shalmali Joshi %E Zachary Lipton %E Rajesh Ranganath %E Iñigo Urteaga %E Serene Yeung %F pmlr-v219-patel23a %I PMLR %P 565--587 %U https://proceedings.mlr.press/v219/patel23a.html %V 219 %X Manual segmentation of brain metastases (BM) is a laborious and time-consuming task for expert clinicians, especially in the setting of longitudinal patient imaging. Although automated deep learning (DL) approaches can segment larger lesions effectively, they suffer from poor sensitivity of lesion detection for micro-metastases. Moreover, these approaches segment all patient imaging independently of each other, ignoring relevant information from prior time-points. In order to utilize prior time-point information, we propose SPIRS, a joint image registration and segmentation method. Given a prior time-point image and segmentation mask (which are readily available in a routine clinical environment), we affinely and deformably register these to a new time-point image. This warped prior image and mask are then used to enhance and improve the segmentation of the new time-point. We apply SPIRS to a large retrospectively acquired single institution dataset and show that it outperforms current registration approaches on BM imaging and that it significantly improves segmentation performance for micro-metastatic lesions.
APA
Patel, J., Ahmed, S.R., Chang, K., Singh, P., Gidwani, M., Hoebel, K., Kim, A., Bridge, C., Teng, C., Li, X., Xu, G., McDonald, M., Aizer, A., Bi, W.L., Ly, I., Rosen, B., Brastianos, P., Huang, R., Gerstner, E. & Kalpathy-Cramer, J.. (2023). A Deep Learning Based Framework for Joint Image Registration and Segmentation of Brain Metastases on Magnetic Resonance Imaging. Proceedings of the 8th Machine Learning for Healthcare Conference, in Proceedings of Machine Learning Research 219:565-587 Available from https://proceedings.mlr.press/v219/patel23a.html.

Related Material