HoVer-NeXt: A Fast Nuclei Segmentation and Classification Pipeline for Next Generation Histopathology

Elias Baumann, Bastian Dislich, Josef Lorenz Rumberger, Iris D. Nagtegaal, Maria Rodriguez Martinez, Inti Zlobec
Proceedings of The 7nd International Conference on Medical Imaging with Deep Learning, PMLR 250:61-86, 2024.

Abstract

In cancer, a variety of cell types, along with their local density and spatial organization within tissues, play a key role in driving cancer progression and modulating patient outcomes. At the basis of cancer diagnosis is the histopathological assessment of tissues, stained by hematoxylin & eosin (H&E), which gives the nuclei of cells a dark purple appearance, making them particularly distinguishable and quantifiable. The identification of individual nuclei, whether in a proliferating (mitosis) or resting state, and their further phenotyping (e.g. immune cells) is the foundation on which histopathology images can be used for further investigations into cellular interaction, prognosis or response prediction. To this end, we develop a H&E based nuclei segmentation and classification model that is both fast (1.8s/mm2 at 0.5mpp, 3.2s/mm2 at 0.25mpp) and accurate (0.84 binary F1, 0.758 mean balanced Accuracy) which allows us to investigate the cellular composition of large-scale colorectal cancer (CRC) cohorts. We extend the publicly available Lizard CRC nuclei dataset with a mitosis class and publish further validation data for the rarest classes: mitosis and eosinophils. Moreover, our pipeline is 5× faster than the CellViT pipeline, 17× faster than the HoVer-Net pipeline, and performs competitively on the PanNuke pan-cancer nuclei dataset (47.7 mPQTiss, +3% over HoVer-Net). Our work paves the way towards extensive single-cell information directly from H&E slides, leading to a quantitative view of whole slide images. Code, model weights as well as all additional training and validation data, are publicly available on github.

Cite this Paper

BibTeX
@InProceedings{pmlr-v250-baumann24a, title = {HoVer-NeXt: A Fast Nuclei Segmentation and Classification Pipeline for Next Generation Histopathology}, author = {Baumann, Elias and Dislich, Bastian and Rumberger, Josef Lorenz and Nagtegaal, Iris D. and Martinez, Maria Rodriguez and Zlobec, Inti}, booktitle = {Proceedings of The 7nd International Conference on Medical Imaging with Deep Learning}, pages = {61--86}, year = {2024}, editor = {Burgos, Ninon and Petitjean, Caroline and Vakalopoulou, Maria and Christodoulidis, Stergios and Coupe, Pierrick and Delingette, Hervé and Lartizien, Carole and Mateus, Diana}, volume = {250}, series = {Proceedings of Machine Learning Research}, month = {03--05 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v250/main/assets/baumann24a/baumann24a.pdf}, url = {https://proceedings.mlr.press/v250/baumann24a.html}, abstract = {In cancer, a variety of cell types, along with their local density and spatial organization within tissues, play a key role in driving cancer progression and modulating patient outcomes. At the basis of cancer diagnosis is the histopathological assessment of tissues, stained by hematoxylin & eosin (H&E), which gives the nuclei of cells a dark purple appearance, making them particularly distinguishable and quantifiable. The identification of individual nuclei, whether in a proliferating (mitosis) or resting state, and their further phenotyping (e.g. immune cells) is the foundation on which histopathology images can be used for further investigations into cellular interaction, prognosis or response prediction. To this end, we develop a H&E based nuclei segmentation and classification model that is both fast (1.8s/mm2 at 0.5mpp, 3.2s/mm2 at 0.25mpp) and accurate (0.84 binary F1, 0.758 mean balanced Accuracy) which allows us to investigate the cellular composition of large-scale colorectal cancer (CRC) cohorts. We extend the publicly available Lizard CRC nuclei dataset with a mitosis class and publish further validation data for the rarest classes: mitosis and eosinophils. Moreover, our pipeline is 5× faster than the CellViT pipeline, 17× faster than the HoVer-Net pipeline, and performs competitively on the PanNuke pan-cancer nuclei dataset (47.7 mPQTiss, +3% over HoVer-Net). Our work paves the way towards extensive single-cell information directly from H&E slides, leading to a quantitative view of whole slide images. Code, model weights as well as all additional training and validation data, are publicly available on github.} }
Endnote
%0 Conference Paper %T HoVer-NeXt: A Fast Nuclei Segmentation and Classification Pipeline for Next Generation Histopathology %A Elias Baumann %A Bastian Dislich %A Josef Lorenz Rumberger %A Iris D. Nagtegaal %A Maria Rodriguez Martinez %A Inti Zlobec %B Proceedings of The 7nd International Conference on Medical Imaging with Deep Learning %C Proceedings of Machine Learning Research %D 2024 %E Ninon Burgos %E Caroline Petitjean %E Maria Vakalopoulou %E Stergios Christodoulidis %E Pierrick Coupe %E Hervé Delingette %E Carole Lartizien %E Diana Mateus %F pmlr-v250-baumann24a %I PMLR %P 61--86 %U https://proceedings.mlr.press/v250/baumann24a.html %V 250 %X In cancer, a variety of cell types, along with their local density and spatial organization within tissues, play a key role in driving cancer progression and modulating patient outcomes. At the basis of cancer diagnosis is the histopathological assessment of tissues, stained by hematoxylin & eosin (H&E), which gives the nuclei of cells a dark purple appearance, making them particularly distinguishable and quantifiable. The identification of individual nuclei, whether in a proliferating (mitosis) or resting state, and their further phenotyping (e.g. immune cells) is the foundation on which histopathology images can be used for further investigations into cellular interaction, prognosis or response prediction. To this end, we develop a H&E based nuclei segmentation and classification model that is both fast (1.8s/mm2 at 0.5mpp, 3.2s/mm2 at 0.25mpp) and accurate (0.84 binary F1, 0.758 mean balanced Accuracy) which allows us to investigate the cellular composition of large-scale colorectal cancer (CRC) cohorts. We extend the publicly available Lizard CRC nuclei dataset with a mitosis class and publish further validation data for the rarest classes: mitosis and eosinophils. Moreover, our pipeline is 5× faster than the CellViT pipeline, 17× faster than the HoVer-Net pipeline, and performs competitively on the PanNuke pan-cancer nuclei dataset (47.7 mPQTiss, +3% over HoVer-Net). Our work paves the way towards extensive single-cell information directly from H&E slides, leading to a quantitative view of whole slide images. Code, model weights as well as all additional training and validation data, are publicly available on github.
APA
Baumann, E., Dislich, B., Rumberger, J.L., Nagtegaal, I.D., Martinez, M.R. & Zlobec, I.. (2024). HoVer-NeXt: A Fast Nuclei Segmentation and Classification Pipeline for Next Generation Histopathology. Proceedings of The 7nd International Conference on Medical Imaging with Deep Learning, in Proceedings of Machine Learning Research 250:61-86 Available from https://proceedings.mlr.press/v250/baumann24a.html.

Related Material