Brain Metastasis Segmentation Network Trained with Robustness to Annotations with Multiple False Negatives

Darvin Yi, Endre Grøvik, Michael Iv, Elizabeth Tong, Greg Zaharchuk, Daniel Rubin
; Proceedings of the Third Conference on Medical Imaging with Deep Learning, PMLR 121:867-880, 2020.

Abstract

Deep learning has proven to be an essential tool for medical image analysis. However, the need for accurately labeled input data, often requiring time- and labor-intensive annotation by experts, is a major limitation to the use of deep learning. One solution to this challenge is to allow for use of coarse or noisy labels, which could permit more efficient and scalable labeling of images. In this work, we develop a lopsided loss function based on entropy regularization that assumes the existence of a nontrivial false negative rate in the target annotations. Starting with a carefully annotated brain metastasis lesion dataset, we simulate data with false negatives by (1) randomly censoring the annotated lesions and (2) systematically censoring the smallest lesions. The latter better models true physician error because smaller lesions are harder to notice than the larger ones. Even with a simulated false negative rate as high as $50%$, applying our loss function to randomly censored data preserves maximum sensitivity at $97%$ of the baseline with uncensored training data, compared to just $10%$ for a standard loss function. For the size-based censorship, performance is restored from $17%$ with the current standard to $88%$ with our lopsided bootstrap loss. Our work will enable more efficient scaling of the image labeling process, in parallel with other approaches on creating more efficient user interfaces and tools for annotation.

Cite this Paper


BibTeX
@InProceedings{pmlr-v121-yi20a, title = {Brain Metastasis Segmentation Network Trained with Robustness to Annotations with Multiple False Negatives}, author = {Yi, Darvin and Gr{\o}vik, Endre and Iv, Michael and Tong, Elizabeth and Zaharchuk, Greg and Rubin, Daniel}, pages = {867--880}, year = {2020}, editor = {Tal Arbel and Ismail Ben Ayed and Marleen de Bruijne and Maxime Descoteaux and Herve Lombaert and Christopher Pal}, volume = {121}, series = {Proceedings of Machine Learning Research}, address = {Montreal, QC, Canada}, month = {06--08 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v121/yi20a/yi20a.pdf}, url = {http://proceedings.mlr.press/v121/yi20a.html}, abstract = {Deep learning has proven to be an essential tool for medical image analysis. However, the need for accurately labeled input data, often requiring time- and labor-intensive annotation by experts, is a major limitation to the use of deep learning. One solution to this challenge is to allow for use of coarse or noisy labels, which could permit more efficient and scalable labeling of images. In this work, we develop a lopsided loss function based on entropy regularization that assumes the existence of a nontrivial false negative rate in the target annotations. Starting with a carefully annotated brain metastasis lesion dataset, we simulate data with false negatives by (1) randomly censoring the annotated lesions and (2) systematically censoring the smallest lesions. The latter better models true physician error because smaller lesions are harder to notice than the larger ones. Even with a simulated false negative rate as high as $50%$, applying our loss function to randomly censored data preserves maximum sensitivity at $97%$ of the baseline with uncensored training data, compared to just $10%$ for a standard loss function. For the size-based censorship, performance is restored from $17%$ with the current standard to $88%$ with our lopsided bootstrap loss. Our work will enable more efficient scaling of the image labeling process, in parallel with other approaches on creating more efficient user interfaces and tools for annotation.} }
Endnote
%0 Conference Paper %T Brain Metastasis Segmentation Network Trained with Robustness to Annotations with Multiple False Negatives %A Darvin Yi %A Endre Grøvik %A Michael Iv %A Elizabeth Tong %A Greg Zaharchuk %A Daniel Rubin %B Proceedings of the Third Conference on Medical Imaging with Deep Learning %C Proceedings of Machine Learning Research %D 2020 %E Tal Arbel %E Ismail Ben Ayed %E Marleen de Bruijne %E Maxime Descoteaux %E Herve Lombaert %E Christopher Pal %F pmlr-v121-yi20a %I PMLR %J Proceedings of Machine Learning Research %P 867--880 %U http://proceedings.mlr.press %V 121 %W PMLR %X Deep learning has proven to be an essential tool for medical image analysis. However, the need for accurately labeled input data, often requiring time- and labor-intensive annotation by experts, is a major limitation to the use of deep learning. One solution to this challenge is to allow for use of coarse or noisy labels, which could permit more efficient and scalable labeling of images. In this work, we develop a lopsided loss function based on entropy regularization that assumes the existence of a nontrivial false negative rate in the target annotations. Starting with a carefully annotated brain metastasis lesion dataset, we simulate data with false negatives by (1) randomly censoring the annotated lesions and (2) systematically censoring the smallest lesions. The latter better models true physician error because smaller lesions are harder to notice than the larger ones. Even with a simulated false negative rate as high as $50%$, applying our loss function to randomly censored data preserves maximum sensitivity at $97%$ of the baseline with uncensored training data, compared to just $10%$ for a standard loss function. For the size-based censorship, performance is restored from $17%$ with the current standard to $88%$ with our lopsided bootstrap loss. Our work will enable more efficient scaling of the image labeling process, in parallel with other approaches on creating more efficient user interfaces and tools for annotation.
APA
Yi, D., Grøvik, E., Iv, M., Tong, E., Zaharchuk, G. & Rubin, D.. (2020). Brain Metastasis Segmentation Network Trained with Robustness to Annotations with Multiple False Negatives. Proceedings of the Third Conference on Medical Imaging with Deep Learning, in PMLR 121:867-880

Related Material