Evaluation of Contrastive Predictive Coding for Histopathology Applications

Karin Stacke, Claes Lundström, Jonas Unger, Gabriel Eilertsen
Proceedings of the Machine Learning for Health NeurIPS Workshop, PMLR 136:328-340, 2020.

Abstract

Recent advances in self-supervised learning for image data are closing the gap between unsupervised and supervised learning. However, the effectiveness of self-supervised methods has primarily been demonstrated for natural images. If the results would extrapolate to histopathology images, there could be significant benefits due to the reduced need for annotated data. In this paper, Contrastive Predictive Coding (CPC), one of the most promising stateof-the-art self-supervised methods, is extensively evaluated on histology data by varying a range of different parameters, including training objective, resolution, and data setup. From the results, we are able to draw important conclusions on the usefulness of CPC for digital pathology. We show strong evidence of the limitations of the learned representation for tumor classification, where only low-level information learned early during training, in the first CPC layers, is used. Furthermore, in our experiments, diversifying the distribution of the dataset (i.e., data from multiple organs or medical centers) does not lead to the model learning a more general representation. This study deepens the understanding of how the CPC model’s objective relates to intrinsic characteristics of histology datasets and will help the development of effective self-supervised methods for histopathology.

Cite this Paper

BibTeX
@InProceedings{pmlr-v136-stacke20a, title = {Evaluation of Contrastive Predictive Coding for Histopathology Applications}, author = {Stacke, Karin and Lundstr\"om, Claes and Unger, Jonas and Eilertsen, Gabriel}, booktitle = {Proceedings of the Machine Learning for Health NeurIPS Workshop}, pages = {328--340}, year = {2020}, editor = {Alsentzer, Emily and McDermott, Matthew B. A. and Falck, Fabian and Sarkar, Suproteem K. and Roy, Subhrajit and Hyland, Stephanie L.}, volume = {136}, series = {Proceedings of Machine Learning Research}, month = {11 Dec}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v136/stacke20a/stacke20a.pdf}, url = {https://proceedings.mlr.press/v136/stacke20a.html}, abstract = {Recent advances in self-supervised learning for image data are closing the gap between unsupervised and supervised learning. However, the effectiveness of self-supervised methods has primarily been demonstrated for natural images. If the results would extrapolate to histopathology images, there could be significant benefits due to the reduced need for annotated data. In this paper, Contrastive Predictive Coding (CPC), one of the most promising stateof-the-art self-supervised methods, is extensively evaluated on histology data by varying a range of different parameters, including training objective, resolution, and data setup. From the results, we are able to draw important conclusions on the usefulness of CPC for digital pathology. We show strong evidence of the limitations of the learned representation for tumor classification, where only low-level information learned early during training, in the first CPC layers, is used. Furthermore, in our experiments, diversifying the distribution of the dataset (i.e., data from multiple organs or medical centers) does not lead to the model learning a more general representation. This study deepens the understanding of how the CPC model’s objective relates to intrinsic characteristics of histology datasets and will help the development of effective self-supervised methods for histopathology.} }
Endnote
%0 Conference Paper %T Evaluation of Contrastive Predictive Coding for Histopathology Applications %A Karin Stacke %A Claes Lundström %A Jonas Unger %A Gabriel Eilertsen %B Proceedings of the Machine Learning for Health NeurIPS Workshop %C Proceedings of Machine Learning Research %D 2020 %E Emily Alsentzer %E Matthew B. A. McDermott %E Fabian Falck %E Suproteem K. Sarkar %E Subhrajit Roy %E Stephanie L. Hyland %F pmlr-v136-stacke20a %I PMLR %P 328--340 %U https://proceedings.mlr.press/v136/stacke20a.html %V 136 %X Recent advances in self-supervised learning for image data are closing the gap between unsupervised and supervised learning. However, the effectiveness of self-supervised methods has primarily been demonstrated for natural images. If the results would extrapolate to histopathology images, there could be significant benefits due to the reduced need for annotated data. In this paper, Contrastive Predictive Coding (CPC), one of the most promising stateof-the-art self-supervised methods, is extensively evaluated on histology data by varying a range of different parameters, including training objective, resolution, and data setup. From the results, we are able to draw important conclusions on the usefulness of CPC for digital pathology. We show strong evidence of the limitations of the learned representation for tumor classification, where only low-level information learned early during training, in the first CPC layers, is used. Furthermore, in our experiments, diversifying the distribution of the dataset (i.e., data from multiple organs or medical centers) does not lead to the model learning a more general representation. This study deepens the understanding of how the CPC model’s objective relates to intrinsic characteristics of histology datasets and will help the development of effective self-supervised methods for histopathology.
APA
Stacke, K., Lundström, C., Unger, J. & Eilertsen, G.. (2020). Evaluation of Contrastive Predictive Coding for Histopathology Applications. Proceedings of the Machine Learning for Health NeurIPS Workshop, in Proceedings of Machine Learning Research 136:328-340 Available from https://proceedings.mlr.press/v136/stacke20a.html.

Related Material