Attributes Shape the Embedding Space of Face Recognition Models

Pierrick Leroy, Antonio Mastropietro, Marco Nurisso, Francesco Vaccarino
Proceedings of the 42nd International Conference on Machine Learning, PMLR 267:33960-33983, 2025.

Abstract

Face Recognition (FR) tasks have made significant progress with the advent of Deep Neural Networks, particularly through margin-based triplet losses that embed facial images into high-dimensional feature spaces. During training, these contrastive losses focus exclusively on identity information as labels. However, we observe a multiscale geometric structure emerging in the embedding space, influenced by interpretable facial (e.g., hair color) and image attributes (e.g., contrast). We propose a geometric approach to describe the dependence or invariance of FR models to these attributes and introduce a physics-inspired alignment metric. We evaluate the proposed metric on controlled, simplified models and widely used FR models fine-tuned with synthetic data for targeted attribute augmentation. Our findings reveal that the models exhibit varying degrees of invariance across different attributes, providing insight into their strengths and weaknesses and enabling deeper interpretability. Code available here: https://github.com/mantonios107/attrs-fr-embs.

Cite this Paper

BibTeX
@InProceedings{pmlr-v267-leroy25a, title = {Attributes Shape the Embedding Space of Face Recognition Models}, author = {Leroy, Pierrick and Mastropietro, Antonio and Nurisso, Marco and Vaccarino, Francesco}, booktitle = {Proceedings of the 42nd International Conference on Machine Learning}, pages = {33960--33983}, year = {2025}, editor = {Singh, Aarti and Fazel, Maryam and Hsu, Daniel and Lacoste-Julien, Simon and Berkenkamp, Felix and Maharaj, Tegan and Wagstaff, Kiri and Zhu, Jerry}, volume = {267}, series = {Proceedings of Machine Learning Research}, month = {13--19 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v267/main/assets/leroy25a/leroy25a.pdf}, url = {https://proceedings.mlr.press/v267/leroy25a.html}, abstract = {Face Recognition (FR) tasks have made significant progress with the advent of Deep Neural Networks, particularly through margin-based triplet losses that embed facial images into high-dimensional feature spaces. During training, these contrastive losses focus exclusively on identity information as labels. However, we observe a multiscale geometric structure emerging in the embedding space, influenced by interpretable facial (e.g., hair color) and image attributes (e.g., contrast). We propose a geometric approach to describe the dependence or invariance of FR models to these attributes and introduce a physics-inspired alignment metric. We evaluate the proposed metric on controlled, simplified models and widely used FR models fine-tuned with synthetic data for targeted attribute augmentation. Our findings reveal that the models exhibit varying degrees of invariance across different attributes, providing insight into their strengths and weaknesses and enabling deeper interpretability. Code available here: https://github.com/mantonios107/attrs-fr-embs.} }
Endnote
%0 Conference Paper %T Attributes Shape the Embedding Space of Face Recognition Models %A Pierrick Leroy %A Antonio Mastropietro %A Marco Nurisso %A Francesco Vaccarino %B Proceedings of the 42nd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2025 %E Aarti Singh %E Maryam Fazel %E Daniel Hsu %E Simon Lacoste-Julien %E Felix Berkenkamp %E Tegan Maharaj %E Kiri Wagstaff %E Jerry Zhu %F pmlr-v267-leroy25a %I PMLR %P 33960--33983 %U https://proceedings.mlr.press/v267/leroy25a.html %V 267 %X Face Recognition (FR) tasks have made significant progress with the advent of Deep Neural Networks, particularly through margin-based triplet losses that embed facial images into high-dimensional feature spaces. During training, these contrastive losses focus exclusively on identity information as labels. However, we observe a multiscale geometric structure emerging in the embedding space, influenced by interpretable facial (e.g., hair color) and image attributes (e.g., contrast). We propose a geometric approach to describe the dependence or invariance of FR models to these attributes and introduce a physics-inspired alignment metric. We evaluate the proposed metric on controlled, simplified models and widely used FR models fine-tuned with synthetic data for targeted attribute augmentation. Our findings reveal that the models exhibit varying degrees of invariance across different attributes, providing insight into their strengths and weaknesses and enabling deeper interpretability. Code available here: https://github.com/mantonios107/attrs-fr-embs.
APA
Leroy, P., Mastropietro, A., Nurisso, M. & Vaccarino, F.. (2025). Attributes Shape the Embedding Space of Face Recognition Models. Proceedings of the 42nd International Conference on Machine Learning, in Proceedings of Machine Learning Research 267:33960-33983 Available from https://proceedings.mlr.press/v267/leroy25a.html.

Related Material