The Self-Supervision Regime and Encoder Fit for Histopathology Image Analysis

Asfandyar Azhar
Proceedings of the 4th Machine Learning for Health Symposium, PMLR 259:48-60, 2025.

Abstract

This study systematically compares vision transformers (ViTs) and residual neural networks (ResNets) for slide-level histopathology analysis using self-supervised learning (SSL) strategies SimCLR and DINO, alongside ImageNet-initialized counterparts. Binary and multiclass classification were evaluated on the TCGA-CRCk and PANDA datasets using the multiple instance learning paradigm. GradCAM and attention rollout were applied for visual explainability for ResNets and ViTs, respectively. The results showed that SimCLR excelled with smaller architectures, while DINO performed better with larger ones. ResNets underperformed with DINO regardless of architecture size, whereas ViTs excelled on the PANDA dataset but required scaling to perform well on TCGA-CRCk, a more complex dataset. An emerging self-supervision regime and encoder fit suggests that DINO-ViTs and SimCLR-ResNets are well-suited pairs, though the effects of SSL methods on encoders remain unclear. ViTs demonstrated superior scalability and interpretability compared to ResNets, but further research is needed to better understand the interplay between SSL methods and encoders.

Cite this Paper

BibTeX
@InProceedings{pmlr-v259-azhar25a, title = {The Self-Supervision Regime and Encoder Fit for Histopathology Image Analysis}, author = {Azhar, Asfandyar}, booktitle = {Proceedings of the 4th Machine Learning for Health Symposium}, pages = {48--60}, year = {2025}, editor = {Hegselmann, Stefan and Zhou, Helen and Healey, Elizabeth and Chang, Trenton and Ellington, Caleb and Mhasawade, Vishwali and Tonekaboni, Sana and Argaw, Peniel and Zhang, Haoran}, volume = {259}, series = {Proceedings of Machine Learning Research}, month = {15--16 Dec}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v259/main/assets/azhar25a/azhar25a.pdf}, url = {https://proceedings.mlr.press/v259/azhar25a.html}, abstract = {This study systematically compares vision transformers (ViTs) and residual neural networks (ResNets) for slide-level histopathology analysis using self-supervised learning (SSL) strategies SimCLR and DINO, alongside ImageNet-initialized counterparts. Binary and multiclass classification were evaluated on the TCGA-CRCk and PANDA datasets using the multiple instance learning paradigm. GradCAM and attention rollout were applied for visual explainability for ResNets and ViTs, respectively. The results showed that SimCLR excelled with smaller architectures, while DINO performed better with larger ones. ResNets underperformed with DINO regardless of architecture size, whereas ViTs excelled on the PANDA dataset but required scaling to perform well on TCGA-CRCk, a more complex dataset. An emerging self-supervision regime and encoder fit suggests that DINO-ViTs and SimCLR-ResNets are well-suited pairs, though the effects of SSL methods on encoders remain unclear. ViTs demonstrated superior scalability and interpretability compared to ResNets, but further research is needed to better understand the interplay between SSL methods and encoders.} }
Endnote
%0 Conference Paper %T The Self-Supervision Regime and Encoder Fit for Histopathology Image Analysis %A Asfandyar Azhar %B Proceedings of the 4th Machine Learning for Health Symposium %C Proceedings of Machine Learning Research %D 2025 %E Stefan Hegselmann %E Helen Zhou %E Elizabeth Healey %E Trenton Chang %E Caleb Ellington %E Vishwali Mhasawade %E Sana Tonekaboni %E Peniel Argaw %E Haoran Zhang %F pmlr-v259-azhar25a %I PMLR %P 48--60 %U https://proceedings.mlr.press/v259/azhar25a.html %V 259 %X This study systematically compares vision transformers (ViTs) and residual neural networks (ResNets) for slide-level histopathology analysis using self-supervised learning (SSL) strategies SimCLR and DINO, alongside ImageNet-initialized counterparts. Binary and multiclass classification were evaluated on the TCGA-CRCk and PANDA datasets using the multiple instance learning paradigm. GradCAM and attention rollout were applied for visual explainability for ResNets and ViTs, respectively. The results showed that SimCLR excelled with smaller architectures, while DINO performed better with larger ones. ResNets underperformed with DINO regardless of architecture size, whereas ViTs excelled on the PANDA dataset but required scaling to perform well on TCGA-CRCk, a more complex dataset. An emerging self-supervision regime and encoder fit suggests that DINO-ViTs and SimCLR-ResNets are well-suited pairs, though the effects of SSL methods on encoders remain unclear. ViTs demonstrated superior scalability and interpretability compared to ResNets, but further research is needed to better understand the interplay between SSL methods and encoders.
APA
Azhar, A.. (2025). The Self-Supervision Regime and Encoder Fit for Histopathology Image Analysis. Proceedings of the 4th Machine Learning for Health Symposium, in Proceedings of Machine Learning Research 259:48-60 Available from https://proceedings.mlr.press/v259/azhar25a.html.

Related Material