Parameter Efficient Fine-Tuning of Segment Anything Model for Biomedical Imaging

Carolin Teuber, Anwai Archit, Constantin Pape
Proceedings of The 8th International Conference on Medical Imaging with Deep Learning, PMLR 301:1508-1549, 2026.

Abstract

Segmentation is an important analysis task for biomedical images, enabling the study of individual organelles, cells or organs. Deep learning has massively improved segmentation methods, but challenges remain in generalization to new conditions, requiring costly data annotation. Vision foundation models, such as Segment Anything Model (SAM), address this issue through improved generalization. However, these models still require finetuning on annotated data, although with less annotations, to achieve optimal results for new conditions. As a downside, they require more computational resources. This makes parameter-efficient finetuning (PEFT) relevant. We contribute the first comprehensive study of PEFT for SAM applied to biomedical images. We find that the placement of PEFT layers is more important for efficiency than the type of layer for vision transformers and we provide a recipe for resource-efficient finetuning.

Cite this Paper

BibTeX
@InProceedings{pmlr-v301-teuber26a, title = {Parameter Efficient Fine-Tuning of Segment Anything Model for Biomedical Imaging}, author = {Teuber, Carolin and Archit, Anwai and Pape, Constantin}, booktitle = {Proceedings of The 8th International Conference on Medical Imaging with Deep Learning}, pages = {1508--1549}, year = {2026}, editor = {Tasdizen, Tolga and Elhabian, Shireen and Summers, Ronald and Chen, Chen and Koch, Lisa and Zhuang, Yan}, volume = {301}, series = {Proceedings of Machine Learning Research}, month = {09--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v301/main/assets/teuber26a/teuber26a.pdf}, url = {https://proceedings.mlr.press/v301/teuber26a.html}, abstract = {Segmentation is an important analysis task for biomedical images, enabling the study of individual organelles, cells or organs. Deep learning has massively improved segmentation methods, but challenges remain in generalization to new conditions, requiring costly data annotation. Vision foundation models, such as Segment Anything Model (SAM), address this issue through improved generalization. However, these models still require finetuning on annotated data, although with less annotations, to achieve optimal results for new conditions. As a downside, they require more computational resources. This makes parameter-efficient finetuning (PEFT) relevant. We contribute the first comprehensive study of PEFT for SAM applied to biomedical images. We find that the placement of PEFT layers is more important for efficiency than the type of layer for vision transformers and we provide a recipe for resource-efficient finetuning.} }
Endnote
%0 Conference Paper %T Parameter Efficient Fine-Tuning of Segment Anything Model for Biomedical Imaging %A Carolin Teuber %A Anwai Archit %A Constantin Pape %B Proceedings of The 8th International Conference on Medical Imaging with Deep Learning %C Proceedings of Machine Learning Research %D 2026 %E Tolga Tasdizen %E Shireen Elhabian %E Ronald Summers %E Chen Chen %E Lisa Koch %E Yan Zhuang %F pmlr-v301-teuber26a %I PMLR %P 1508--1549 %U https://proceedings.mlr.press/v301/teuber26a.html %V 301 %X Segmentation is an important analysis task for biomedical images, enabling the study of individual organelles, cells or organs. Deep learning has massively improved segmentation methods, but challenges remain in generalization to new conditions, requiring costly data annotation. Vision foundation models, such as Segment Anything Model (SAM), address this issue through improved generalization. However, these models still require finetuning on annotated data, although with less annotations, to achieve optimal results for new conditions. As a downside, they require more computational resources. This makes parameter-efficient finetuning (PEFT) relevant. We contribute the first comprehensive study of PEFT for SAM applied to biomedical images. We find that the placement of PEFT layers is more important for efficiency than the type of layer for vision transformers and we provide a recipe for resource-efficient finetuning.
APA
Teuber, C., Archit, A. & Pape, C.. (2026). Parameter Efficient Fine-Tuning of Segment Anything Model for Biomedical Imaging. Proceedings of The 8th International Conference on Medical Imaging with Deep Learning, in Proceedings of Machine Learning Research 301:1508-1549 Available from https://proceedings.mlr.press/v301/teuber26a.html.

Related Material