What Fine-Tuning Changes: A Radiomic Lens on Prostate Foundation Model Representations

Yipei Wang, Yaxi Chen, Wen Yan, Natasha Thorley, Alexander Ng, Dean C. Barratt, Daniel C. Alexander, Shonit Punwani, Mark Emberton, Veeru Kasivisvanathan, Yipeng Hu
Proceedings of The 9th International Conference on Medical Imaging with Deep Learning, PMLR 315:2233-2247, 2026.

Abstract

Clarifying how foundation model encoders change during fine-tuning is important for transparency and trustworthiness in their medical imaging applications. It may also be useful for further understanding, developing, and adapting these models. However, the latent representations produced by such encoders are high dimensional and lack explicit semantic meaning, making it difficult to characterise how task-specific adaptation modifies them. In this study, we introduce a radiomics-based framework that provides an interpretable lens through which these representational changes can be examined and often better understood. Using prostate cancer patient imaging data, we train a two-layer MLP to learn the relationship between radiomic descriptors and encoder embeddings prior to fine-tuning. This model captures non-linear associations through its first layer, while the final linear layer offers an interpretable mapping from radiomic attributes to (transformed) latent features. To quantify the effect of fine-tuning, the first layer is fixed, and only the linear layer is re-estimated using the embeddings from the fine-tuned encoder. Comparing the pre- and post-fine-tuning linear weights yields a direct quantitative measure of how the encoder’s emphasis on specific radiomic characteristics shifts during fine-tuning. We validate the approach using a prostate MRI foundation model and multiple downstream tasks. The analysis reveals consistent, task-dependent changes in the encoder’s sensitivity to radiomic texture and intensity features. This work provides the first radiomics-based methodology for systematically interpreting how fine-tuning restructures foundation model representation in medical imaging.

Cite this Paper

BibTeX
@InProceedings{pmlr-v315-wang26e, title = {What Fine-Tuning Changes: A Radiomic Lens on Prostate Foundation Model Representations}, author = {Wang, Yipei and Chen, Yaxi and Yan, Wen and Thorley, Natasha and Ng, Alexander and Barratt, Dean C. and Alexander, Daniel C. and Punwani, Shonit and Emberton, Mark and Kasivisvanathan, Veeru and Hu, Yipeng}, booktitle = {Proceedings of The 9th International Conference on Medical Imaging with Deep Learning}, pages = {2233--2247}, year = {2026}, editor = {Huo, Yuankai and Gao, Mingchen and Kuo, Chang-Fu and Jin, Yueming and Deng, Ruining}, volume = {315}, series = {Proceedings of Machine Learning Research}, month = {08--10 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v315/main/assets/wang26e/wang26e.pdf}, url = {https://proceedings.mlr.press/v315/wang26e.html}, abstract = {Clarifying how foundation model encoders change during fine-tuning is important for transparency and trustworthiness in their medical imaging applications. It may also be useful for further understanding, developing, and adapting these models. However, the latent representations produced by such encoders are high dimensional and lack explicit semantic meaning, making it difficult to characterise how task-specific adaptation modifies them. In this study, we introduce a radiomics-based framework that provides an interpretable lens through which these representational changes can be examined and often better understood. Using prostate cancer patient imaging data, we train a two-layer MLP to learn the relationship between radiomic descriptors and encoder embeddings prior to fine-tuning. This model captures non-linear associations through its first layer, while the final linear layer offers an interpretable mapping from radiomic attributes to (transformed) latent features. To quantify the effect of fine-tuning, the first layer is fixed, and only the linear layer is re-estimated using the embeddings from the fine-tuned encoder. Comparing the pre- and post-fine-tuning linear weights yields a direct quantitative measure of how the encoder’s emphasis on specific radiomic characteristics shifts during fine-tuning. We validate the approach using a prostate MRI foundation model and multiple downstream tasks. The analysis reveals consistent, task-dependent changes in the encoder’s sensitivity to radiomic texture and intensity features. This work provides the first radiomics-based methodology for systematically interpreting how fine-tuning restructures foundation model representation in medical imaging.} }
Endnote
%0 Conference Paper %T What Fine-Tuning Changes: A Radiomic Lens on Prostate Foundation Model Representations %A Yipei Wang %A Yaxi Chen %A Wen Yan %A Natasha Thorley %A Alexander Ng %A Dean C. Barratt %A Daniel C. Alexander %A Shonit Punwani %A Mark Emberton %A Veeru Kasivisvanathan %A Yipeng Hu %B Proceedings of The 9th International Conference on Medical Imaging with Deep Learning %C Proceedings of Machine Learning Research %D 2026 %E Yuankai Huo %E Mingchen Gao %E Chang-Fu Kuo %E Yueming Jin %E Ruining Deng %F pmlr-v315-wang26e %I PMLR %P 2233--2247 %U https://proceedings.mlr.press/v315/wang26e.html %V 315 %X Clarifying how foundation model encoders change during fine-tuning is important for transparency and trustworthiness in their medical imaging applications. It may also be useful for further understanding, developing, and adapting these models. However, the latent representations produced by such encoders are high dimensional and lack explicit semantic meaning, making it difficult to characterise how task-specific adaptation modifies them. In this study, we introduce a radiomics-based framework that provides an interpretable lens through which these representational changes can be examined and often better understood. Using prostate cancer patient imaging data, we train a two-layer MLP to learn the relationship between radiomic descriptors and encoder embeddings prior to fine-tuning. This model captures non-linear associations through its first layer, while the final linear layer offers an interpretable mapping from radiomic attributes to (transformed) latent features. To quantify the effect of fine-tuning, the first layer is fixed, and only the linear layer is re-estimated using the embeddings from the fine-tuned encoder. Comparing the pre- and post-fine-tuning linear weights yields a direct quantitative measure of how the encoder’s emphasis on specific radiomic characteristics shifts during fine-tuning. We validate the approach using a prostate MRI foundation model and multiple downstream tasks. The analysis reveals consistent, task-dependent changes in the encoder’s sensitivity to radiomic texture and intensity features. This work provides the first radiomics-based methodology for systematically interpreting how fine-tuning restructures foundation model representation in medical imaging.
APA
Wang, Y., Chen, Y., Yan, W., Thorley, N., Ng, A., Barratt, D.C., Alexander, D.C., Punwani, S., Emberton, M., Kasivisvanathan, V. & Hu, Y.. (2026). What Fine-Tuning Changes: A Radiomic Lens on Prostate Foundation Model Representations. Proceedings of The 9th International Conference on Medical Imaging with Deep Learning, in Proceedings of Machine Learning Research 315:2233-2247 Available from https://proceedings.mlr.press/v315/wang26e.html.

Related Material