Nuc2Vec: Learning Representations of Nuclei in Histopathology Images with Contrastive Loss

Chao Feng, Chad Vanderbilt, Thomas Fuchs
Proceedings of the Fourth Conference on Medical Imaging with Deep Learning, PMLR 143:179-189, 2021.

Abstract

The tumor microenvironment is an area of intense interest in cancer research and may be a clinically actionable aspect of cancer care. One way to study the tumor microenvironment is to characterize the spatial interactions between various types of nuclei in cancer tissue from H&E whole slide images, which require nucleus segmentation and classification. Current methods of nucleus classification rely on extensive labeling from pathologists and are limited by the number of categories a nucleus can be classified into. In this work, leveraging existing nucleus segmentation and contrastive representation learning methods, we developed a method that learns vector embeddings of nuclei based on their morphology in histopathology images. We show that the embeddings learned by this model capture distinctive morphological features of nuclei and can be used to group them into meaningful subtypes. These embeddings can provide a much richer characterization of the statistics of the spatial distribution of nuclei in cancer and open new possibilities in the quantitative study of the tumor microenvironment.

Cite this Paper

BibTeX
@InProceedings{pmlr-v143-feng21a, title = {Nuc2Vec: Learning Representations of Nuclei in Histopathology Images with Contrastive Loss}, author = {Feng, Chao and Vanderbilt, Chad and Fuchs, Thomas}, booktitle = {Proceedings of the Fourth Conference on Medical Imaging with Deep Learning}, pages = {179--189}, year = {2021}, editor = {Heinrich, Mattias and Dou, Qi and de Bruijne, Marleen and Lellmann, Jan and Schläfer, Alexander and Ernst, Floris}, volume = {143}, series = {Proceedings of Machine Learning Research}, month = {07--09 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v143/feng21a/feng21a.pdf}, url = {https://proceedings.mlr.press/v143/feng21a.html}, abstract = {The tumor microenvironment is an area of intense interest in cancer research and may be a clinically actionable aspect of cancer care. One way to study the tumor microenvironment is to characterize the spatial interactions between various types of nuclei in cancer tissue from H&E whole slide images, which require nucleus segmentation and classification. Current methods of nucleus classification rely on extensive labeling from pathologists and are limited by the number of categories a nucleus can be classified into. In this work, leveraging existing nucleus segmentation and contrastive representation learning methods, we developed a method that learns vector embeddings of nuclei based on their morphology in histopathology images. We show that the embeddings learned by this model capture distinctive morphological features of nuclei and can be used to group them into meaningful subtypes. These embeddings can provide a much richer characterization of the statistics of the spatial distribution of nuclei in cancer and open new possibilities in the quantitative study of the tumor microenvironment.} }
Endnote
%0 Conference Paper %T Nuc2Vec: Learning Representations of Nuclei in Histopathology Images with Contrastive Loss %A Chao Feng %A Chad Vanderbilt %A Thomas Fuchs %B Proceedings of the Fourth Conference on Medical Imaging with Deep Learning %C Proceedings of Machine Learning Research %D 2021 %E Mattias Heinrich %E Qi Dou %E Marleen de Bruijne %E Jan Lellmann %E Alexander Schläfer %E Floris Ernst %F pmlr-v143-feng21a %I PMLR %P 179--189 %U https://proceedings.mlr.press/v143/feng21a.html %V 143 %X The tumor microenvironment is an area of intense interest in cancer research and may be a clinically actionable aspect of cancer care. One way to study the tumor microenvironment is to characterize the spatial interactions between various types of nuclei in cancer tissue from H&E whole slide images, which require nucleus segmentation and classification. Current methods of nucleus classification rely on extensive labeling from pathologists and are limited by the number of categories a nucleus can be classified into. In this work, leveraging existing nucleus segmentation and contrastive representation learning methods, we developed a method that learns vector embeddings of nuclei based on their morphology in histopathology images. We show that the embeddings learned by this model capture distinctive morphological features of nuclei and can be used to group them into meaningful subtypes. These embeddings can provide a much richer characterization of the statistics of the spatial distribution of nuclei in cancer and open new possibilities in the quantitative study of the tumor microenvironment.
APA
Feng, C., Vanderbilt, C. & Fuchs, T.. (2021). Nuc2Vec: Learning Representations of Nuclei in Histopathology Images with Contrastive Loss. Proceedings of the Fourth Conference on Medical Imaging with Deep Learning, in Proceedings of Machine Learning Research 143:179-189 Available from https://proceedings.mlr.press/v143/feng21a.html.

Related Material