PathologyGAN: Learning deep representations of cancer tissue

Adalberto Claudio Quiros, Roderick Murray-Smith, Ke Yuan
; Proceedings of the Third Conference on Medical Imaging with Deep Learning, PMLR 121:669-695, 2020.

Abstract

We apply Generative Adversarial Networks (GANs) to the domain of digital pathology. Current machine learning research for digital pathology focuses on diagnosis, but we suggest a different approach and advocate that generative models could drive forward the understanding of morphological characteristics of cancer tissue. In this paper, we develop a framework which allows GANs to capture key tissue features and uses these characteristics to give structure to its latent space. To this end, we trained our model on $249$K H$&$E breast cancer tissue images, extracted from 576 TMA images of patients from the Netherlands Cancer Institute (NKI) and Vancouver General Hospital (VGH) cohorts. We show that our model generates high quality images, with a Fréchet Inception Distance (FID) of 16.65. We further assess the quality of the images with cancer tissue characteristics (e.g. count of cancer, lymphocytes, or stromal cells), using quantitative information to calculate the FID and showing consistent performance of 9.86. Additionally, the latent space of our model shows an interpretable structure and allows semantic vector operations that translate into tissue feature transformations. Furthermore, ratings from two expert pathologists found no significant difference between our generated tissue images from real ones. The code, generated images, and pretrained model are available at \href{https://github.com/AdalbertoCq/Pathology-GAN}{https://github.com/AdalbertoCq/Pathology-GAN}

Cite this Paper


BibTeX
@InProceedings{pmlr-v121-quiros20a, title = {PathologyGAN: Learning deep representations of cancer tissue}, author = {Quiros, Adalberto Claudio and Murray-Smith, Roderick and Yuan, Ke}, pages = {669--695}, year = {2020}, editor = {Tal Arbel and Ismail Ben Ayed and Marleen de Bruijne and Maxime Descoteaux and Herve Lombaert and Christopher Pal}, volume = {121}, series = {Proceedings of Machine Learning Research}, address = {Montreal, QC, Canada}, month = {06--08 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v121/quiros20a/quiros20a.pdf}, url = {http://proceedings.mlr.press/v121/quiros20a.html}, abstract = {We apply Generative Adversarial Networks (GANs) to the domain of digital pathology. Current machine learning research for digital pathology focuses on diagnosis, but we suggest a different approach and advocate that generative models could drive forward the understanding of morphological characteristics of cancer tissue. In this paper, we develop a framework which allows GANs to capture key tissue features and uses these characteristics to give structure to its latent space. To this end, we trained our model on $249$K H$&$E breast cancer tissue images, extracted from 576 TMA images of patients from the Netherlands Cancer Institute (NKI) and Vancouver General Hospital (VGH) cohorts. We show that our model generates high quality images, with a Fréchet Inception Distance (FID) of 16.65. We further assess the quality of the images with cancer tissue characteristics (e.g. count of cancer, lymphocytes, or stromal cells), using quantitative information to calculate the FID and showing consistent performance of 9.86. Additionally, the latent space of our model shows an interpretable structure and allows semantic vector operations that translate into tissue feature transformations. Furthermore, ratings from two expert pathologists found no significant difference between our generated tissue images from real ones. The code, generated images, and pretrained model are available at \href{https://github.com/AdalbertoCq/Pathology-GAN}{https://github.com/AdalbertoCq/Pathology-GAN}} }
Endnote
%0 Conference Paper %T PathologyGAN: Learning deep representations of cancer tissue %A Adalberto Claudio Quiros %A Roderick Murray-Smith %A Ke Yuan %B Proceedings of the Third Conference on Medical Imaging with Deep Learning %C Proceedings of Machine Learning Research %D 2020 %E Tal Arbel %E Ismail Ben Ayed %E Marleen de Bruijne %E Maxime Descoteaux %E Herve Lombaert %E Christopher Pal %F pmlr-v121-quiros20a %I PMLR %J Proceedings of Machine Learning Research %P 669--695 %U http://proceedings.mlr.press %V 121 %W PMLR %X We apply Generative Adversarial Networks (GANs) to the domain of digital pathology. Current machine learning research for digital pathology focuses on diagnosis, but we suggest a different approach and advocate that generative models could drive forward the understanding of morphological characteristics of cancer tissue. In this paper, we develop a framework which allows GANs to capture key tissue features and uses these characteristics to give structure to its latent space. To this end, we trained our model on $249$K H$&$E breast cancer tissue images, extracted from 576 TMA images of patients from the Netherlands Cancer Institute (NKI) and Vancouver General Hospital (VGH) cohorts. We show that our model generates high quality images, with a Fréchet Inception Distance (FID) of 16.65. We further assess the quality of the images with cancer tissue characteristics (e.g. count of cancer, lymphocytes, or stromal cells), using quantitative information to calculate the FID and showing consistent performance of 9.86. Additionally, the latent space of our model shows an interpretable structure and allows semantic vector operations that translate into tissue feature transformations. Furthermore, ratings from two expert pathologists found no significant difference between our generated tissue images from real ones. The code, generated images, and pretrained model are available at \href{https://github.com/AdalbertoCq/Pathology-GAN}{https://github.com/AdalbertoCq/Pathology-GAN}
APA
Quiros, A.C., Murray-Smith, R. & Yuan, K.. (2020). PathologyGAN: Learning deep representations of cancer tissue. Proceedings of the Third Conference on Medical Imaging with Deep Learning, in PMLR 121:669-695

Related Material