Zero-shot capability of 2D SAM-family models for bone segmentation in CT scans

Caroline Magg, Clara I. Sánchez, Hoel Kervadec
Proceedings of The 8th International Conference on Medical Imaging with Deep Learning, PMLR 301:1053-1073, 2026.

Abstract

The Segment Anything Model (Sam) and similar models build a family of promptable foundation models (FMs) for image and video segmentation. The object of interest is identified using prompts—user provided input such as bounding boxes or points—and the models have shown very promising results when it comes to generalization to new tasks.However, extensive evaluation studies are required for medical applications, to assess their strengths and weaknesses in clinical settings.As the performance of those models is highly dependent on the quality and quantity of their prompts, it is necessary to thoroughly benchmark the different options. Currently, no dedicated evaluation studies exist specifically for bone segmentation in CT scans. Leveraging high-quality private and public datasets on four skeletal regions, we test the zero-shot capabilities of SAM-family models for bone CT segmentation, using non-interactive prompting strategies, composed of bounding box, points and combinations of the two. Additionally, we design a guideline for informed decision-making in 2D non-interactive prompting based on our insights on segmentation performance and inference time.Our results show that SAM and SAM2 currently outperform medically fine-tuned FMs, and prompted with a bounding box together with a center point have the best performance across all tested settings. Our code is available in this github repository (https://github.com/CarolineMagg/SAM-family-2D-benchmark).

Cite this Paper

BibTeX
@InProceedings{pmlr-v301-magg26a, title = {Zero-shot capability of 2D SAM-family models for bone segmentation in CT scans}, author = {Magg, Caroline and S\'anchez, Clara I. and Kervadec, Hoel}, booktitle = {Proceedings of The 8th International Conference on Medical Imaging with Deep Learning}, pages = {1053--1073}, year = {2026}, editor = {Tasdizen, Tolga and Elhabian, Shireen and Summers, Ronald and Chen, Chen and Koch, Lisa and Zhuang, Yan}, volume = {301}, series = {Proceedings of Machine Learning Research}, month = {09--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v301/main/assets/magg26a/magg26a.pdf}, url = {https://proceedings.mlr.press/v301/magg26a.html}, abstract = {The Segment Anything Model (Sam) and similar models build a family of promptable foundation models (FMs) for image and video segmentation. The object of interest is identified using prompts—user provided input such as bounding boxes or points—and the models have shown very promising results when it comes to generalization to new tasks.However, extensive evaluation studies are required for medical applications, to assess their strengths and weaknesses in clinical settings.As the performance of those models is highly dependent on the quality and quantity of their prompts, it is necessary to thoroughly benchmark the different options. Currently, no dedicated evaluation studies exist specifically for bone segmentation in CT scans. Leveraging high-quality private and public datasets on four skeletal regions, we test the zero-shot capabilities of SAM-family models for bone CT segmentation, using non-interactive prompting strategies, composed of bounding box, points and combinations of the two. Additionally, we design a guideline for informed decision-making in 2D non-interactive prompting based on our insights on segmentation performance and inference time.Our results show that SAM and SAM2 currently outperform medically fine-tuned FMs, and prompted with a bounding box together with a center point have the best performance across all tested settings. Our code is available in this github repository (https://github.com/CarolineMagg/SAM-family-2D-benchmark).} }
Endnote
%0 Conference Paper %T Zero-shot capability of 2D SAM-family models for bone segmentation in CT scans %A Caroline Magg %A Clara I. Sánchez %A Hoel Kervadec %B Proceedings of The 8th International Conference on Medical Imaging with Deep Learning %C Proceedings of Machine Learning Research %D 2026 %E Tolga Tasdizen %E Shireen Elhabian %E Ronald Summers %E Chen Chen %E Lisa Koch %E Yan Zhuang %F pmlr-v301-magg26a %I PMLR %P 1053--1073 %U https://proceedings.mlr.press/v301/magg26a.html %V 301 %X The Segment Anything Model (Sam) and similar models build a family of promptable foundation models (FMs) for image and video segmentation. The object of interest is identified using prompts—user provided input such as bounding boxes or points—and the models have shown very promising results when it comes to generalization to new tasks.However, extensive evaluation studies are required for medical applications, to assess their strengths and weaknesses in clinical settings.As the performance of those models is highly dependent on the quality and quantity of their prompts, it is necessary to thoroughly benchmark the different options. Currently, no dedicated evaluation studies exist specifically for bone segmentation in CT scans. Leveraging high-quality private and public datasets on four skeletal regions, we test the zero-shot capabilities of SAM-family models for bone CT segmentation, using non-interactive prompting strategies, composed of bounding box, points and combinations of the two. Additionally, we design a guideline for informed decision-making in 2D non-interactive prompting based on our insights on segmentation performance and inference time.Our results show that SAM and SAM2 currently outperform medically fine-tuned FMs, and prompted with a bounding box together with a center point have the best performance across all tested settings. Our code is available in this github repository (https://github.com/CarolineMagg/SAM-family-2D-benchmark).
APA
Magg, C., Sánchez, C.I. & Kervadec, H.. (2026). Zero-shot capability of 2D SAM-family models for bone segmentation in CT scans. Proceedings of The 8th International Conference on Medical Imaging with Deep Learning, in Proceedings of Machine Learning Research 301:1053-1073 Available from https://proceedings.mlr.press/v301/magg26a.html.

Related Material