Zero-Shot Medical Image Segmentation Based on Sparse Prompt Using Finetuned SAM

Tal Shaharabany, Lior Wolf
Proceedings of The 7nd International Conference on Medical Imaging with Deep Learning, PMLR 250:1387-1400, 2024.

Abstract

Segmentation of medical images plays a critical role in various clinical applications, facilitat- ing precise diagnosis, treatment planning, and disease monitoring. However, the scarcity of annotated data poses a significant challenge for training deep learning models in the medical imaging domain. In this paper, we propose a novel approach for minimally-guided zero-shot segmentation of medical images using the Segment Anything Model (SAM), orig- inally trained on natural images. The method leverages SAM’s ability to segment arbitrary objects in natural scenes and adapts it to the medical domain without the need for labeled medical data, except for a few foreground and background points on the test image it- self. To this end, we introduce a two-stage process, involving the extraction of an initial mask from self-similarity maps and test-time fine-tuning of SAM. We run experiments on diverse medical imaging datasets, including AMOS22, MoNuSeg and the Gland segmen- tation (GlaS) challenge, and demonstrate the effectiveness of our approach.

Cite this Paper

BibTeX
@InProceedings{pmlr-v250-shaharabany24a, title = {Zero-Shot Medical Image Segmentation Based on Sparse Prompt Using Finetuned SAM}, author = {Shaharabany, Tal and Wolf, Lior}, booktitle = {Proceedings of The 7nd International Conference on Medical Imaging with Deep Learning}, pages = {1387--1400}, year = {2024}, editor = {Burgos, Ninon and Petitjean, Caroline and Vakalopoulou, Maria and Christodoulidis, Stergios and Coupe, Pierrick and Delingette, Hervé and Lartizien, Carole and Mateus, Diana}, volume = {250}, series = {Proceedings of Machine Learning Research}, month = {03--05 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v250/main/assets/shaharabany24a/shaharabany24a.pdf}, url = {https://proceedings.mlr.press/v250/shaharabany24a.html}, abstract = {Segmentation of medical images plays a critical role in various clinical applications, facilitat- ing precise diagnosis, treatment planning, and disease monitoring. However, the scarcity of annotated data poses a significant challenge for training deep learning models in the medical imaging domain. In this paper, we propose a novel approach for minimally-guided zero-shot segmentation of medical images using the Segment Anything Model (SAM), orig- inally trained on natural images. The method leverages SAM’s ability to segment arbitrary objects in natural scenes and adapts it to the medical domain without the need for labeled medical data, except for a few foreground and background points on the test image it- self. To this end, we introduce a two-stage process, involving the extraction of an initial mask from self-similarity maps and test-time fine-tuning of SAM. We run experiments on diverse medical imaging datasets, including AMOS22, MoNuSeg and the Gland segmen- tation (GlaS) challenge, and demonstrate the effectiveness of our approach.} }
Endnote
%0 Conference Paper %T Zero-Shot Medical Image Segmentation Based on Sparse Prompt Using Finetuned SAM %A Tal Shaharabany %A Lior Wolf %B Proceedings of The 7nd International Conference on Medical Imaging with Deep Learning %C Proceedings of Machine Learning Research %D 2024 %E Ninon Burgos %E Caroline Petitjean %E Maria Vakalopoulou %E Stergios Christodoulidis %E Pierrick Coupe %E Hervé Delingette %E Carole Lartizien %E Diana Mateus %F pmlr-v250-shaharabany24a %I PMLR %P 1387--1400 %U https://proceedings.mlr.press/v250/shaharabany24a.html %V 250 %X Segmentation of medical images plays a critical role in various clinical applications, facilitat- ing precise diagnosis, treatment planning, and disease monitoring. However, the scarcity of annotated data poses a significant challenge for training deep learning models in the medical imaging domain. In this paper, we propose a novel approach for minimally-guided zero-shot segmentation of medical images using the Segment Anything Model (SAM), orig- inally trained on natural images. The method leverages SAM’s ability to segment arbitrary objects in natural scenes and adapts it to the medical domain without the need for labeled medical data, except for a few foreground and background points on the test image it- self. To this end, we introduce a two-stage process, involving the extraction of an initial mask from self-similarity maps and test-time fine-tuning of SAM. We run experiments on diverse medical imaging datasets, including AMOS22, MoNuSeg and the Gland segmen- tation (GlaS) challenge, and demonstrate the effectiveness of our approach.
APA
Shaharabany, T. & Wolf, L.. (2024). Zero-Shot Medical Image Segmentation Based on Sparse Prompt Using Finetuned SAM. Proceedings of The 7nd International Conference on Medical Imaging with Deep Learning, in Proceedings of Machine Learning Research 250:1387-1400 Available from https://proceedings.mlr.press/v250/shaharabany24a.html.

Related Material