Brain Metastasis Segmentation Network Trained with Robustness to Annotations with Multiple False Negatives

Darvin Yi, Endre Grøvik, Michael Iv, Elizabeth Tong, Greg Zaharchuk, Daniel Rubin
Proceedings of the Third Conference on Medical Imaging with Deep Learning, PMLR 121:867-880, 2020.

Abstract

Deep learning has proven to be an essential tool for medical image analysis. However, the need for accurately labeled input data, often requiring time- and labor-intensive annotation by experts, is a major limitation to the use of deep learning. One solution to this challenge is to allow for use of coarse or noisy labels, which could permit more efficient and scalable labeling of images. In this work, we develop a lopsided loss function based on entropy regularization that assumes the existence of a nontrivial false negative rate in the target annotations. Starting with a carefully annotated brain metastasis lesion dataset, we simulate data with false negatives by (1) randomly censoring the annotated lesions and (2) systematically censoring the smallest lesions. The latter better models true physician error because smaller lesions are harder to notice than the larger ones. Even with a simulated false negative rate as high as $50%$, applying our loss function to randomly censored data preserves maximum sensitivity at $97%$ of the baseline with uncensored training data, compared to just $10%$ for a standard loss function. For the size-based censorship, performance is restored from $17%$ with the current standard to $88%$ with our lopsided bootstrap loss. Our work will enable more efficient scaling of the image labeling process, in parallel with other approaches on creating more efficient user interfaces and tools for annotation.

Cite this Paper

BibTeX
@InProceedings{pmlr-v121-yi20a, title = {Brain Metastasis Segmentation Network Trained with Robustness to Annotations with Multiple False Negatives}, author = {Yi, Darvin and Gr{\o}vik, Endre and Iv, Michael and Tong, Elizabeth and Zaharchuk, Greg and Rubin, Daniel}, booktitle = {Proceedings of the Third Conference on Medical Imaging with Deep Learning}, pages = {867--880}, year = {2020}, editor = {Arbel, Tal and Ben Ayed, Ismail and de Bruijne, Marleen and Descoteaux, Maxime and Lombaert, Herve and Pal, Christopher}, volume = {121}, series = {Proceedings of Machine Learning Research}, month = {06--08 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v121/yi20a/yi20a.pdf}, url = {https://proceedings.mlr.press/v121/yi20a.html}, abstract = {Deep learning has proven to be an essential tool for medical image analysis. However, the need for accurately labeled input data, often requiring time- and labor-intensive annotation by experts, is a major limitation to the use of deep learning. One solution to this challenge is to allow for use of coarse or noisy labels, which could permit more efficient and scalable labeling of images. In this work, we develop a lopsided loss function based on entropy regularization that assumes the existence of a nontrivial false negative rate in the target annotations. Starting with a carefully annotated brain metastasis lesion dataset, we simulate data with false negatives by (1) randomly censoring the annotated lesions and (2) systematically censoring the smallest lesions. The latter better models true physician error because smaller lesions are harder to notice than the larger ones. Even with a simulated false negative rate as high as $50%$, applying our loss function to randomly censored data preserves maximum sensitivity at $97%$ of the baseline with uncensored training data, compared to just $10%$ for a standard loss function. For the size-based censorship, performance is restored from $17%$ with the current standard to $88%$ with our lopsided bootstrap loss. Our work will enable more efficient scaling of the image labeling process, in parallel with other approaches on creating more efficient user interfaces and tools for annotation.} }
Endnote
%0 Conference Paper %T Brain Metastasis Segmentation Network Trained with Robustness to Annotations with Multiple False Negatives %A Darvin Yi %A Endre Grøvik %A Michael Iv %A Elizabeth Tong %A Greg Zaharchuk %A Daniel Rubin %B Proceedings of the Third Conference on Medical Imaging with Deep Learning %C Proceedings of Machine Learning Research %D 2020 %E Tal Arbel %E Ismail Ben Ayed %E Marleen de Bruijne %E Maxime Descoteaux %E Herve Lombaert %E Christopher Pal %F pmlr-v121-yi20a %I PMLR %P 867--880 %U https://proceedings.mlr.press/v121/yi20a.html %V 121 %X Deep learning has proven to be an essential tool for medical image analysis. However, the need for accurately labeled input data, often requiring time- and labor-intensive annotation by experts, is a major limitation to the use of deep learning. One solution to this challenge is to allow for use of coarse or noisy labels, which could permit more efficient and scalable labeling of images. In this work, we develop a lopsided loss function based on entropy regularization that assumes the existence of a nontrivial false negative rate in the target annotations. Starting with a carefully annotated brain metastasis lesion dataset, we simulate data with false negatives by (1) randomly censoring the annotated lesions and (2) systematically censoring the smallest lesions. The latter better models true physician error because smaller lesions are harder to notice than the larger ones. Even with a simulated false negative rate as high as $50%$, applying our loss function to randomly censored data preserves maximum sensitivity at $97%$ of the baseline with uncensored training data, compared to just $10%$ for a standard loss function. For the size-based censorship, performance is restored from $17%$ with the current standard to $88%$ with our lopsided bootstrap loss. Our work will enable more efficient scaling of the image labeling process, in parallel with other approaches on creating more efficient user interfaces and tools for annotation.
APA
Yi, D., Grøvik, E., Iv, M., Tong, E., Zaharchuk, G. & Rubin, D.. (2020). Brain Metastasis Segmentation Network Trained with Robustness to Annotations with Multiple False Negatives. Proceedings of the Third Conference on Medical Imaging with Deep Learning, in Proceedings of Machine Learning Research 121:867-880 Available from https://proceedings.mlr.press/v121/yi20a.html.

Related Material