Well-Calibrated Regression Uncertainty in Medical Imaging with Deep Learning

Max-Heinrich Laves, Sontje Ihler, Jacob F. Fast, Lüder A. Kahrs, Tobias Ortmaier
Proceedings of the Third Conference on Medical Imaging with Deep Learning, PMLR 121:393-412, 2020.

Abstract

The consideration of predictive uncertainty in medical imaging with deep learning is of utmost importance. We apply estimation of predictive uncertainty by variational Bayesian inference with Monte Carlo dropout to regression tasks and show why predictive uncertainty is systematically underestimated. We suggest using $ \sigma $ {\em scaling} with a single scalar value; a simple, yet effective calibration method for both aleatoric and epistemic uncertainty. The performance of our approach is evaluated on a variety of common medical regression data sets using different state-of-the-art convolutional network architectures. In all experiments, $\sigma $ scaling is able to reliably recalibrate predictive uncertainty. It is easy to implement and maintains the accuracy. Well-calibrated uncertainty in regression allows robust rejection of unreliable predictions or detection of out-of-distribution samples. Our source code is available at: {https://github.com/mlaves/well-calibrated-regression-uncertainty}

Cite this Paper

BibTeX
@InProceedings{pmlr-v121-laves20a, title = {Well-Calibrated Regression Uncertainty in Medical Imaging with Deep Learning}, author = {Laves, Max-Heinrich and Ihler, Sontje and Fast, Jacob F. and Kahrs, L\"uder A. and Ortmaier, Tobias}, booktitle = {Proceedings of the Third Conference on Medical Imaging with Deep Learning}, pages = {393--412}, year = {2020}, editor = {Arbel, Tal and Ben Ayed, Ismail and de Bruijne, Marleen and Descoteaux, Maxime and Lombaert, Herve and Pal, Christopher}, volume = {121}, series = {Proceedings of Machine Learning Research}, month = {06--08 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v121/laves20a/laves20a.pdf}, url = {https://proceedings.mlr.press/v121/laves20a.html}, abstract = {The consideration of predictive uncertainty in medical imaging with deep learning is of utmost importance. We apply estimation of predictive uncertainty by variational Bayesian inference with Monte Carlo dropout to regression tasks and show why predictive uncertainty is systematically underestimated. We suggest using $ \sigma $ {\em scaling} with a single scalar value; a simple, yet effective calibration method for both aleatoric and epistemic uncertainty. The performance of our approach is evaluated on a variety of common medical regression data sets using different state-of-the-art convolutional network architectures. In all experiments, $\sigma $ scaling is able to reliably recalibrate predictive uncertainty. It is easy to implement and maintains the accuracy. Well-calibrated uncertainty in regression allows robust rejection of unreliable predictions or detection of out-of-distribution samples. Our source code is available at: {https://github.com/mlaves/well-calibrated-regression-uncertainty}} }
Endnote
%0 Conference Paper %T Well-Calibrated Regression Uncertainty in Medical Imaging with Deep Learning %A Max-Heinrich Laves %A Sontje Ihler %A Jacob F. Fast %A Lüder A. Kahrs %A Tobias Ortmaier %B Proceedings of the Third Conference on Medical Imaging with Deep Learning %C Proceedings of Machine Learning Research %D 2020 %E Tal Arbel %E Ismail Ben Ayed %E Marleen de Bruijne %E Maxime Descoteaux %E Herve Lombaert %E Christopher Pal %F pmlr-v121-laves20a %I PMLR %P 393--412 %U https://proceedings.mlr.press/v121/laves20a.html %V 121 %X The consideration of predictive uncertainty in medical imaging with deep learning is of utmost importance. We apply estimation of predictive uncertainty by variational Bayesian inference with Monte Carlo dropout to regression tasks and show why predictive uncertainty is systematically underestimated. We suggest using $ \sigma $ {\em scaling} with a single scalar value; a simple, yet effective calibration method for both aleatoric and epistemic uncertainty. The performance of our approach is evaluated on a variety of common medical regression data sets using different state-of-the-art convolutional network architectures. In all experiments, $\sigma $ scaling is able to reliably recalibrate predictive uncertainty. It is easy to implement and maintains the accuracy. Well-calibrated uncertainty in regression allows robust rejection of unreliable predictions or detection of out-of-distribution samples. Our source code is available at: {https://github.com/mlaves/well-calibrated-regression-uncertainty}
APA
Laves, M., Ihler, S., Fast, J.F., Kahrs, L.A. & Ortmaier, T.. (2020). Well-Calibrated Regression Uncertainty in Medical Imaging with Deep Learning. Proceedings of the Third Conference on Medical Imaging with Deep Learning, in Proceedings of Machine Learning Research 121:393-412 Available from https://proceedings.mlr.press/v121/laves20a.html.

Related Material