A Benchmark of Foundation Model Encoders for Histopathological Image Segmentation

Itsaso Vitoria, Cristina L. Saratxaga, Cristina Penas Lago, Rosa Izu, Ana Sanchez-Diez, Goikoana Cancho-Galan, Maria Dolores Boyano, Ignacio Arganda-Carreras, Adrian Galdran
Proceedings of the MICCAI Workshop on Computational Pathology, PMLR 316:199-212, 2026.

Abstract

Whole-slide imaging has transformed histopathology into a data-intensive field, requiring robust and generalisable computational tools. Foundation models offer a promising approach for a range of downstream tasks with minimal labelled data. While recent work has shown their effectiveness for slide-level classification and retrieval, their potential for dense prediction tasks such as image segmentation remains underexplored. In this study, we present a comprehensive benchmark of 15 pathology-specific foundation models for histopathological image segmentation, evaluated across two distinct modalities: H&Estained histology and Annexin A5-stained immunohistochemistry. To ensure a fair and architecture-neutral comparison, we freeze each foundation models encoder and pair it with a shared lightweight decoder, disentangling representation quality from model size. Results show that foundation model encoders can sometimes lead to strong segmentation performance without fine-tuning, but effectiveness varies significantly by model and modality. Our findings reveal that compact encoders can often outperform larger, more recent models, underscoring that model size and classification accuracy are poor predictors of segmentation capabilities.

Cite this Paper

BibTeX
@InProceedings{pmlr-v316-vitoria26a, title = {A Benchmark of Foundation Model Encoders for Histopathological Image Segmentation}, author = {Vitoria, Itsaso and Saratxaga, Cristina L. and Lago, Cristina Penas and Izu, Rosa and Sanchez-Diez, Ana and Cancho-Galan, Goikoana and Boyano, Maria Dolores and Arganda-Carreras, Ignacio and Galdran, Adrian}, booktitle = {Proceedings of the MICCAI Workshop on Computational Pathology}, pages = {199--212}, year = {2026}, editor = {Studer, Linda and Ciompi, Francesco and Khalili, Nadieh and Faryna, Khrystyna and Faryna, Khrystyna and Yeong, Joe and Lau, Mai Chan and Chen, Hao and Liu, Ziyi and Brattoli, Biagio}, volume = {316}, series = {Proceedings of Machine Learning Research}, month = {27 Sep}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v316/main/assets/vitoria26a/vitoria26a.pdf}, url = {https://proceedings.mlr.press/v316/vitoria26a.html}, abstract = {Whole-slide imaging has transformed histopathology into a data-intensive field, requiring robust and generalisable computational tools. Foundation models offer a promising approach for a range of downstream tasks with minimal labelled data. While recent work has shown their effectiveness for slide-level classification and retrieval, their potential for dense prediction tasks such as image segmentation remains underexplored. In this study, we present a comprehensive benchmark of 15 pathology-specific foundation models for histopathological image segmentation, evaluated across two distinct modalities: H&Estained histology and Annexin A5-stained immunohistochemistry. To ensure a fair and architecture-neutral comparison, we freeze each foundation models encoder and pair it with a shared lightweight decoder, disentangling representation quality from model size. Results show that foundation model encoders can sometimes lead to strong segmentation performance without fine-tuning, but effectiveness varies significantly by model and modality. Our findings reveal that compact encoders can often outperform larger, more recent models, underscoring that model size and classification accuracy are poor predictors of segmentation capabilities.} }
Endnote
%0 Conference Paper %T A Benchmark of Foundation Model Encoders for Histopathological Image Segmentation %A Itsaso Vitoria %A Cristina L. Saratxaga %A Cristina Penas Lago %A Rosa Izu %A Ana Sanchez-Diez %A Goikoana Cancho-Galan %A Maria Dolores Boyano %A Ignacio Arganda-Carreras %A Adrian Galdran %B Proceedings of the MICCAI Workshop on Computational Pathology %C Proceedings of Machine Learning Research %D 2026 %E Linda Studer %E Francesco Ciompi %E Nadieh Khalili %E Khrystyna Faryna %E Khrystyna Faryna %E Joe Yeong %E Mai Chan Lau %E Hao Chen %E Ziyi Liu %E Biagio Brattoli %F pmlr-v316-vitoria26a %I PMLR %P 199--212 %U https://proceedings.mlr.press/v316/vitoria26a.html %V 316 %X Whole-slide imaging has transformed histopathology into a data-intensive field, requiring robust and generalisable computational tools. Foundation models offer a promising approach for a range of downstream tasks with minimal labelled data. While recent work has shown their effectiveness for slide-level classification and retrieval, their potential for dense prediction tasks such as image segmentation remains underexplored. In this study, we present a comprehensive benchmark of 15 pathology-specific foundation models for histopathological image segmentation, evaluated across two distinct modalities: H&Estained histology and Annexin A5-stained immunohistochemistry. To ensure a fair and architecture-neutral comparison, we freeze each foundation models encoder and pair it with a shared lightweight decoder, disentangling representation quality from model size. Results show that foundation model encoders can sometimes lead to strong segmentation performance without fine-tuning, but effectiveness varies significantly by model and modality. Our findings reveal that compact encoders can often outperform larger, more recent models, underscoring that model size and classification accuracy are poor predictors of segmentation capabilities.
APA
Vitoria, I., Saratxaga, C.L., Lago, C.P., Izu, R., Sanchez-Diez, A., Cancho-Galan, G., Boyano, M.D., Arganda-Carreras, I. & Galdran, A.. (2026). A Benchmark of Foundation Model Encoders for Histopathological Image Segmentation. Proceedings of the MICCAI Workshop on Computational Pathology, in Proceedings of Machine Learning Research 316:199-212 Available from https://proceedings.mlr.press/v316/vitoria26a.html.

Related Material