Impact of uncertainty maps on manual editing of rectal cancer segmentation in radiotherapy

Federica Carmen Maruccio, Rita Simões, Fokie Cnossen, Christian Jamtheim Gustafsson, Sanne Conijn, Alice Couwenberg, Suzan Gerrets-van Noord, Inge de Jong, Vivian van Pelt, Lisa Wiersema, Joëlle van Aalst, Jan-Jakob Sonke, Charlotte L. Brouwer, Tomas Janssen
Proceedings of The 9th International Conference on Medical Imaging with Deep Learning, PMLR 315:4401-4447, 2026.

Abstract

Uncertainty maps provide a quantitative and visual representation of the estimated confidence of Deep Learning (DL) models in contouring predictions and have been proposed to improve clinicians’ efficiency during manual review. However, uncertainty maps are not currently integrated into clinical workflows, and evidence on their actual benefit in clinical decision-making remains limited. This study investigates the impact of simulated uncertainty maps on clinicians’ behaviour during manual editing of high-quality clinical target volume (CTV) contours in rectal cancer radiotherapy. An inter-observer variability dataset of ten patients was used to simulate meaningful DL uncertainty maps and contours. Six clinicians edited the contours across two editing sessions, with and without uncertainty maps. For each session, editing time, editing amount, questionnaire responses, and interview feedback were collected to assess the impact both quantitatively and qualitatively. Editing time and editing amount were comparable with and without uncertainty maps, while both measures decreased significantly in the second editing session, indicating a learning effect from task repetition. Qualitative feedback showed that clinicians’ decisions were shaped more by human factors, such as workload, mood, memory and anchoring biases, than by the uncertainty maps. Moreover, the study revealed low clinician trust in the uncertainty maps, which were used primarily for confirmation rather than decision-making. The findings suggest that the value of uncertainty maps may be limited for high-quality contours and highlight the need to investigate their relevance for different use cases.

Cite this Paper

BibTeX
@InProceedings{pmlr-v315-maruccio26a, title = {Impact of uncertainty maps on manual editing of rectal cancer segmentation in radiotherapy}, author = {Maruccio, Federica Carmen and Sim\~oes, Rita and Cnossen, Fokie and Jamtheim Gustafsson, Christian and Conijn, Sanne and Couwenberg, Alice and Gerrets-van Noord, Suzan and de Jong, Inge and van Pelt, Vivian and Wiersema, Lisa and van Aalst, Jo\"elle and Sonke, Jan-Jakob and Brouwer, Charlotte L. and Janssen, Tomas}, booktitle = {Proceedings of The 9th International Conference on Medical Imaging with Deep Learning}, pages = {4401--4447}, year = {2026}, editor = {Huo, Yuankai and Gao, Mingchen and Kuo, Chang-Fu and Jin, Yueming and Deng, Ruining}, volume = {315}, series = {Proceedings of Machine Learning Research}, month = {08--10 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v315/main/assets/maruccio26a/maruccio26a.pdf}, url = {https://proceedings.mlr.press/v315/maruccio26a.html}, abstract = {Uncertainty maps provide a quantitative and visual representation of the estimated confidence of Deep Learning (DL) models in contouring predictions and have been proposed to improve clinicians’ efficiency during manual review. However, uncertainty maps are not currently integrated into clinical workflows, and evidence on their actual benefit in clinical decision-making remains limited. This study investigates the impact of simulated uncertainty maps on clinicians’ behaviour during manual editing of high-quality clinical target volume (CTV) contours in rectal cancer radiotherapy. An inter-observer variability dataset of ten patients was used to simulate meaningful DL uncertainty maps and contours. Six clinicians edited the contours across two editing sessions, with and without uncertainty maps. For each session, editing time, editing amount, questionnaire responses, and interview feedback were collected to assess the impact both quantitatively and qualitatively. Editing time and editing amount were comparable with and without uncertainty maps, while both measures decreased significantly in the second editing session, indicating a learning effect from task repetition. Qualitative feedback showed that clinicians’ decisions were shaped more by human factors, such as workload, mood, memory and anchoring biases, than by the uncertainty maps. Moreover, the study revealed low clinician trust in the uncertainty maps, which were used primarily for confirmation rather than decision-making. The findings suggest that the value of uncertainty maps may be limited for high-quality contours and highlight the need to investigate their relevance for different use cases.} }
Endnote
%0 Conference Paper %T Impact of uncertainty maps on manual editing of rectal cancer segmentation in radiotherapy %A Federica Carmen Maruccio %A Rita Simões %A Fokie Cnossen %A Christian Jamtheim Gustafsson %A Sanne Conijn %A Alice Couwenberg %A Suzan Gerrets-van Noord %A Inge de Jong %A Vivian van Pelt %A Lisa Wiersema %A Joëlle van Aalst %A Jan-Jakob Sonke %A Charlotte L. Brouwer %A Tomas Janssen %B Proceedings of The 9th International Conference on Medical Imaging with Deep Learning %C Proceedings of Machine Learning Research %D 2026 %E Yuankai Huo %E Mingchen Gao %E Chang-Fu Kuo %E Yueming Jin %E Ruining Deng %F pmlr-v315-maruccio26a %I PMLR %P 4401--4447 %U https://proceedings.mlr.press/v315/maruccio26a.html %V 315 %X Uncertainty maps provide a quantitative and visual representation of the estimated confidence of Deep Learning (DL) models in contouring predictions and have been proposed to improve clinicians’ efficiency during manual review. However, uncertainty maps are not currently integrated into clinical workflows, and evidence on their actual benefit in clinical decision-making remains limited. This study investigates the impact of simulated uncertainty maps on clinicians’ behaviour during manual editing of high-quality clinical target volume (CTV) contours in rectal cancer radiotherapy. An inter-observer variability dataset of ten patients was used to simulate meaningful DL uncertainty maps and contours. Six clinicians edited the contours across two editing sessions, with and without uncertainty maps. For each session, editing time, editing amount, questionnaire responses, and interview feedback were collected to assess the impact both quantitatively and qualitatively. Editing time and editing amount were comparable with and without uncertainty maps, while both measures decreased significantly in the second editing session, indicating a learning effect from task repetition. Qualitative feedback showed that clinicians’ decisions were shaped more by human factors, such as workload, mood, memory and anchoring biases, than by the uncertainty maps. Moreover, the study revealed low clinician trust in the uncertainty maps, which were used primarily for confirmation rather than decision-making. The findings suggest that the value of uncertainty maps may be limited for high-quality contours and highlight the need to investigate their relevance for different use cases.
APA
Maruccio, F.C., Simões, R., Cnossen, F., Jamtheim Gustafsson, C., Conijn, S., Couwenberg, A., Gerrets-van Noord, S., de Jong, I., van Pelt, V., Wiersema, L., van Aalst, J., Sonke, J., Brouwer, C.L. & Janssen, T.. (2026). Impact of uncertainty maps on manual editing of rectal cancer segmentation in radiotherapy. Proceedings of The 9th International Conference on Medical Imaging with Deep Learning, in Proceedings of Machine Learning Research 315:4401-4447 Available from https://proceedings.mlr.press/v315/maruccio26a.html.

Related Material