Grounding Functional Similarity by Invariance-Aware Model Stitching

Ioannis Athanasiadis, Anmar Karmush, Michael Felsberg
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:4283-4304, 2026.

Abstract

In deep learning, functional similarity evaluation quantifies the extent to which independently trained models learn similar input–output relationships. In model stitching, functional similarity is framed as representation forward compatibility, i.e., whether the representations of two models can be aligned to solve a given task. Recent studies, however, highlight a critical limitation: models relying on different information cues can still produce compatible representations, making them appear misleadingly similar (Smith et al., 2025). We attribute this failure to standard model stitching being inherently blind to the invariance properties of the stitched models. To address this limitation, we introduce the forward–backward compatibility requirement under which we formulate the invariance-aware model stitching. Through analyzing key stitching configurations, we study the interplay between forward and backward compatibility, showing that invariance-aware model stitching provides a more principled approach to functional similarity evaluation while revealing functional discrepancies previously obscured.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-athanasiadis26a, title = {Grounding Functional Similarity by Invariance-Aware Model Stitching}, author = {Athanasiadis, Ioannis and Karmush, Anmar and Felsberg, Michael}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {4283--4304}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/athanasiadis26a/athanasiadis26a.pdf}, url = {https://proceedings.mlr.press/v306/athanasiadis26a.html}, abstract = {In deep learning, functional similarity evaluation quantifies the extent to which independently trained models learn similar input–output relationships. In model stitching, functional similarity is framed as representation forward compatibility, i.e., whether the representations of two models can be aligned to solve a given task. Recent studies, however, highlight a critical limitation: models relying on different information cues can still produce compatible representations, making them appear misleadingly similar (Smith et al., 2025). We attribute this failure to standard model stitching being inherently blind to the invariance properties of the stitched models. To address this limitation, we introduce the forward–backward compatibility requirement under which we formulate the invariance-aware model stitching. Through analyzing key stitching configurations, we study the interplay between forward and backward compatibility, showing that invariance-aware model stitching provides a more principled approach to functional similarity evaluation while revealing functional discrepancies previously obscured.} }
Endnote
%0 Conference Paper %T Grounding Functional Similarity by Invariance-Aware Model Stitching %A Ioannis Athanasiadis %A Anmar Karmush %A Michael Felsberg %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-athanasiadis26a %I PMLR %P 4283--4304 %U https://proceedings.mlr.press/v306/athanasiadis26a.html %V 306 %X In deep learning, functional similarity evaluation quantifies the extent to which independently trained models learn similar input–output relationships. In model stitching, functional similarity is framed as representation forward compatibility, i.e., whether the representations of two models can be aligned to solve a given task. Recent studies, however, highlight a critical limitation: models relying on different information cues can still produce compatible representations, making them appear misleadingly similar (Smith et al., 2025). We attribute this failure to standard model stitching being inherently blind to the invariance properties of the stitched models. To address this limitation, we introduce the forward–backward compatibility requirement under which we formulate the invariance-aware model stitching. Through analyzing key stitching configurations, we study the interplay between forward and backward compatibility, showing that invariance-aware model stitching provides a more principled approach to functional similarity evaluation while revealing functional discrepancies previously obscured.
APA
Athanasiadis, I., Karmush, A. & Felsberg, M.. (2026). Grounding Functional Similarity by Invariance-Aware Model Stitching. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:4283-4304 Available from https://proceedings.mlr.press/v306/athanasiadis26a.html.

Related Material