Structure Size as Confounder in Uncertainty Based Segmentation Quality Prediction

Kai Geißler, Jochen G. Hirsch, Stefan Heldmann, Hans Meine
Proceedings of The 7nd International Conference on Medical Imaging with Deep Learning, PMLR 250:504-519, 2024.

Abstract

Various uncertainty estimation methods have been proposed for deep learning-based image segmentation models. An uncertainty measure is treated useful if it can be used to accurately predict segmentation quality. Therefore, structure-wise uncertainty measures are frequently correlated with measures like the Dice score. However, it is known that the Dice score highly depends on the size of the structure of interest. It is less well-known that popular structure-wise uncertainty measures also correlate with structure size. Therefore, the structure size acts as confounding variable when trying to quantify the performance of such uncertainty measures via correlation. We investigate this for the popular uncertainty measures structure-wise epistemic uncertainty, mean pairwise Dice and volume variation coefficient based on test-time-augmentation, Monte Carlo Dropout and model ensembles. We propose to use a partial correlation coefficient to address structure size as confounding variable and arrive at lower correlation estimates which better reflect the true relationship between segmentation quality and structure-wise uncertainty.

Cite this Paper

BibTeX
@InProceedings{pmlr-v250-geissler24a, title = {Structure Size as Confounder in Uncertainty Based Segmentation Quality Prediction}, author = {Gei{\ss}ler, Kai and Hirsch, Jochen G. and Heldmann, Stefan and Meine, Hans}, booktitle = {Proceedings of The 7nd International Conference on Medical Imaging with Deep Learning}, pages = {504--519}, year = {2024}, editor = {Burgos, Ninon and Petitjean, Caroline and Vakalopoulou, Maria and Christodoulidis, Stergios and Coupe, Pierrick and Delingette, Hervé and Lartizien, Carole and Mateus, Diana}, volume = {250}, series = {Proceedings of Machine Learning Research}, month = {03--05 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v250/main/assets/geissler24a/geissler24a.pdf}, url = {https://proceedings.mlr.press/v250/geissler24a.html}, abstract = {Various uncertainty estimation methods have been proposed for deep learning-based image segmentation models. An uncertainty measure is treated useful if it can be used to accurately predict segmentation quality. Therefore, structure-wise uncertainty measures are frequently correlated with measures like the Dice score. However, it is known that the Dice score highly depends on the size of the structure of interest. It is less well-known that popular structure-wise uncertainty measures also correlate with structure size. Therefore, the structure size acts as confounding variable when trying to quantify the performance of such uncertainty measures via correlation. We investigate this for the popular uncertainty measures structure-wise epistemic uncertainty, mean pairwise Dice and volume variation coefficient based on test-time-augmentation, Monte Carlo Dropout and model ensembles. We propose to use a partial correlation coefficient to address structure size as confounding variable and arrive at lower correlation estimates which better reflect the true relationship between segmentation quality and structure-wise uncertainty.} }
Endnote
%0 Conference Paper %T Structure Size as Confounder in Uncertainty Based Segmentation Quality Prediction %A Kai Geißler %A Jochen G. Hirsch %A Stefan Heldmann %A Hans Meine %B Proceedings of The 7nd International Conference on Medical Imaging with Deep Learning %C Proceedings of Machine Learning Research %D 2024 %E Ninon Burgos %E Caroline Petitjean %E Maria Vakalopoulou %E Stergios Christodoulidis %E Pierrick Coupe %E Hervé Delingette %E Carole Lartizien %E Diana Mateus %F pmlr-v250-geissler24a %I PMLR %P 504--519 %U https://proceedings.mlr.press/v250/geissler24a.html %V 250 %X Various uncertainty estimation methods have been proposed for deep learning-based image segmentation models. An uncertainty measure is treated useful if it can be used to accurately predict segmentation quality. Therefore, structure-wise uncertainty measures are frequently correlated with measures like the Dice score. However, it is known that the Dice score highly depends on the size of the structure of interest. It is less well-known that popular structure-wise uncertainty measures also correlate with structure size. Therefore, the structure size acts as confounding variable when trying to quantify the performance of such uncertainty measures via correlation. We investigate this for the popular uncertainty measures structure-wise epistemic uncertainty, mean pairwise Dice and volume variation coefficient based on test-time-augmentation, Monte Carlo Dropout and model ensembles. We propose to use a partial correlation coefficient to address structure size as confounding variable and arrive at lower correlation estimates which better reflect the true relationship between segmentation quality and structure-wise uncertainty.
APA
Geißler, K., Hirsch, J.G., Heldmann, S. & Meine, H.. (2024). Structure Size as Confounder in Uncertainty Based Segmentation Quality Prediction. Proceedings of The 7nd International Conference on Medical Imaging with Deep Learning, in Proceedings of Machine Learning Research 250:504-519 Available from https://proceedings.mlr.press/v250/geissler24a.html.

Related Material