Improving Mask R-CNN for Nuclei Instance Segmentation in Hematoxylin & Eosin-Stained Histological Images

Benjamin Bancher, Amirreza Mahbod, Isabella Ellinger, Rupert Ecker, Georg Dorffner
Proceedings of the MICCAI Workshop on Computational Pathology, PMLR 156:20-35, 2021.

Abstract

Digital pathology is an emerging topic in the analysis of pathologic tissue samples. It includes providing the tools towards more automated workflows to derive clinically relevant information. Digitization and storage of whole slide images have become commonplace and allow modern image analysis methods to be used. In recent years, computer-based segmentation of cell nuclei has gathered considerable attention in the development of highly specialized algorithms. Currently, most of these algorithms are based on performing semantic segmentation of all cell nuclei and separating overlapping instances in a post-processing step. Recently, instance-aware segmentation methods such as Mask R-CNN have been proposed to enable unified instance detection and segmentation, even in overlapping cases. In this work, we propose a modified Mask R-CNN-based approach by incorporating distance maps of instances and hematoxylin-stain intensities as extra input channels to the model. Moreover, we explore the impact of three well-known inference strategies, namely test-time augmentation, ensembling, and knowledge transfer through pre-training on the segmentation performance. We perform extensive ablation experiments across multiple runs to quantitatively define the most optimal inference strategy in the proposed Mask R-CNN algorithm. Our results show that average instance segmentation improvements of up to 3.5% and 4.1% based on Aggregate Jaccard Index and Panoptic Quality score can be obtained, respectively, using the proposed techniques in comparison to a standard Mask R-CNN model. Our findings confirm the effectiveness of aggregating information at the network input stage and optimizing inference workflows using minimal effort. Implemented modifications and codes are publicly available through a GitHub repository under: https://github.com/bbanc/Improved-Mask-R-CNN-for-nuclei-segmentation

Cite this Paper

BibTeX
@InProceedings{pmlr-v156-bancher21a, title = {Improving Mask R-CNN for Nuclei Instance Segmentation in Hematoxylin & Eosin-Stained Histological Images}, author = {Bancher, Benjamin and Mahbod, Amirreza and Ellinger, Isabella and Ecker, Rupert and Dorffner, Georg}, booktitle = {Proceedings of the MICCAI Workshop on Computational Pathology}, pages = {20--35}, year = {2021}, editor = {Atzori, Manfredo and Burlutskiy, Nikolay and Ciompi, Francesco and Li, Zhang and Minhas, Fayyaz and Müller, Henning and Peng, Tingying and Rajpoot, Nasir and Torben-Nielsen, Ben and van der Laak, Jeroen and Veta, Mitko and Yuan, Yinyin and Zlobec, Inti}, volume = {156}, series = {Proceedings of Machine Learning Research}, month = {27 Sep}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v156/bancher21a/bancher21a.pdf}, url = {https://proceedings.mlr.press/v156/bancher21a.html}, abstract = {Digital pathology is an emerging topic in the analysis of pathologic tissue samples. It includes providing the tools towards more automated workflows to derive clinically relevant information. Digitization and storage of whole slide images have become commonplace and allow modern image analysis methods to be used. In recent years, computer-based segmentation of cell nuclei has gathered considerable attention in the development of highly specialized algorithms. Currently, most of these algorithms are based on performing semantic segmentation of all cell nuclei and separating overlapping instances in a post-processing step. Recently, instance-aware segmentation methods such as Mask R-CNN have been proposed to enable unified instance detection and segmentation, even in overlapping cases. In this work, we propose a modified Mask R-CNN-based approach by incorporating distance maps of instances and hematoxylin-stain intensities as extra input channels to the model. Moreover, we explore the impact of three well-known inference strategies, namely test-time augmentation, ensembling, and knowledge transfer through pre-training on the segmentation performance. We perform extensive ablation experiments across multiple runs to quantitatively define the most optimal inference strategy in the proposed Mask R-CNN algorithm. Our results show that average instance segmentation improvements of up to 3.5% and 4.1% based on Aggregate Jaccard Index and Panoptic Quality score can be obtained, respectively, using the proposed techniques in comparison to a standard Mask R-CNN model. Our findings confirm the effectiveness of aggregating information at the network input stage and optimizing inference workflows using minimal effort. Implemented modifications and codes are publicly available through a GitHub repository under: https://github.com/bbanc/Improved-Mask-R-CNN-for-nuclei-segmentation} }
Endnote
%0 Conference Paper %T Improving Mask R-CNN for Nuclei Instance Segmentation in Hematoxylin & Eosin-Stained Histological Images %A Benjamin Bancher %A Amirreza Mahbod %A Isabella Ellinger %A Rupert Ecker %A Georg Dorffner %B Proceedings of the MICCAI Workshop on Computational Pathology %C Proceedings of Machine Learning Research %D 2021 %E Manfredo Atzori %E Nikolay Burlutskiy %E Francesco Ciompi %E Zhang Li %E Fayyaz Minhas %E Henning Müller %E Tingying Peng %E Nasir Rajpoot %E Ben Torben-Nielsen %E Jeroen van der Laak %E Mitko Veta %E Yinyin Yuan %E Inti Zlobec %F pmlr-v156-bancher21a %I PMLR %P 20--35 %U https://proceedings.mlr.press/v156/bancher21a.html %V 156 %X Digital pathology is an emerging topic in the analysis of pathologic tissue samples. It includes providing the tools towards more automated workflows to derive clinically relevant information. Digitization and storage of whole slide images have become commonplace and allow modern image analysis methods to be used. In recent years, computer-based segmentation of cell nuclei has gathered considerable attention in the development of highly specialized algorithms. Currently, most of these algorithms are based on performing semantic segmentation of all cell nuclei and separating overlapping instances in a post-processing step. Recently, instance-aware segmentation methods such as Mask R-CNN have been proposed to enable unified instance detection and segmentation, even in overlapping cases. In this work, we propose a modified Mask R-CNN-based approach by incorporating distance maps of instances and hematoxylin-stain intensities as extra input channels to the model. Moreover, we explore the impact of three well-known inference strategies, namely test-time augmentation, ensembling, and knowledge transfer through pre-training on the segmentation performance. We perform extensive ablation experiments across multiple runs to quantitatively define the most optimal inference strategy in the proposed Mask R-CNN algorithm. Our results show that average instance segmentation improvements of up to 3.5% and 4.1% based on Aggregate Jaccard Index and Panoptic Quality score can be obtained, respectively, using the proposed techniques in comparison to a standard Mask R-CNN model. Our findings confirm the effectiveness of aggregating information at the network input stage and optimizing inference workflows using minimal effort. Implemented modifications and codes are publicly available through a GitHub repository under: https://github.com/bbanc/Improved-Mask-R-CNN-for-nuclei-segmentation
APA
Bancher, B., Mahbod, A., Ellinger, I., Ecker, R. & Dorffner, G.. (2021). Improving Mask R-CNN for Nuclei Instance Segmentation in Hematoxylin & Eosin-Stained Histological Images. Proceedings of the MICCAI Workshop on Computational Pathology, in Proceedings of Machine Learning Research 156:20-35 Available from https://proceedings.mlr.press/v156/bancher21a.html.

Related Material