When Algorithms Play Favorites: Lookism in the Generation and Perception of Faces

Miriam Doh, Aditya Gulati, Matei Mancas, Nuria Oliver
Proceedings of Fourth European Workshop on Algorithmic Fairness, PMLR 294:474-480, 2025.

Abstract

This paper examines how synthetically generated faces and machine learning-based gender classification algorithms are affected by algorithmic lookism, the preferential treatment based on appearance. In experiments with 13,200 synthetically generated faces, we find that: (1) text-to-image (T2I) systems tend to associate facial attractiveness to unrelated positive traits like intelligence and trustworthiness; and (2) gender classification models exhibit higher error rates on "less-attractive" faces, especially among non-White women. These result raise fairness concerns regarding digital identity systems.

Cite this Paper

BibTeX
@InProceedings{pmlr-v294-doh25a, title = {When Algorithms Play Favorites: Lookism in the Generation and Perception of Faces}, author = {Doh, Miriam and Gulati, Aditya and Mancas, Matei and Oliver, Nuria}, booktitle = {Proceedings of Fourth European Workshop on Algorithmic Fairness}, pages = {474--480}, year = {2025}, editor = {Weerts, Hilde and Pechenizkiy, Mykola and Allhutter, Doris and Corrêa, Ana Maria and Grote, Thomas and Liem, Cynthia}, volume = {294}, series = {Proceedings of Machine Learning Research}, month = {30 Jun--02 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v294/main/assets/doh25a/doh25a.pdf}, url = {https://proceedings.mlr.press/v294/doh25a.html}, abstract = {This paper examines how synthetically generated faces and machine learning-based gender classification algorithms are affected by algorithmic lookism, the preferential treatment based on appearance. In experiments with 13,200 synthetically generated faces, we find that: (1) text-to-image (T2I) systems tend to associate facial attractiveness to unrelated positive traits like intelligence and trustworthiness; and (2) gender classification models exhibit higher error rates on "less-attractive" faces, especially among non-White women. These result raise fairness concerns regarding digital identity systems.} }
Endnote
%0 Conference Paper %T When Algorithms Play Favorites: Lookism in the Generation and Perception of Faces %A Miriam Doh %A Aditya Gulati %A Matei Mancas %A Nuria Oliver %B Proceedings of Fourth European Workshop on Algorithmic Fairness %C Proceedings of Machine Learning Research %D 2025 %E Hilde Weerts %E Mykola Pechenizkiy %E Doris Allhutter %E Ana Maria Corrêa %E Thomas Grote %E Cynthia Liem %F pmlr-v294-doh25a %I PMLR %P 474--480 %U https://proceedings.mlr.press/v294/doh25a.html %V 294 %X This paper examines how synthetically generated faces and machine learning-based gender classification algorithms are affected by algorithmic lookism, the preferential treatment based on appearance. In experiments with 13,200 synthetically generated faces, we find that: (1) text-to-image (T2I) systems tend to associate facial attractiveness to unrelated positive traits like intelligence and trustworthiness; and (2) gender classification models exhibit higher error rates on "less-attractive" faces, especially among non-White women. These result raise fairness concerns regarding digital identity systems.
APA
Doh, M., Gulati, A., Mancas, M. & Oliver, N.. (2025). When Algorithms Play Favorites: Lookism in the Generation and Perception of Faces. Proceedings of Fourth European Workshop on Algorithmic Fairness, in Proceedings of Machine Learning Research 294:474-480 Available from https://proceedings.mlr.press/v294/doh25a.html.

Related Material