Probing Protected Attributes within General-Purpose LLMs for Hiring

Jan Grenzebach, Thea Radüntz
Proceedings of Fifth European Conference on Algorithmic Fairness, PMLR 350:372-385, 2026.

Abstract

General-purpose Large Language Models (LLM) applications in human resource workflows are primarily driven by the need for objectivity and efficiency. However, a significant gap exists between enterprise-level AI and the consumer-grade ”black-box” services frequently utilized by small and medium-sized enterprises (SMEs), raising critical questions regarding algorithmic consistency and fairness. This paper utilizes the IN-OUT framework, a stimulus-response methodology derived from the cognitive sciences, to empirically audit the behavior of one popular LLM. We employ a 2 $\times$ 2 factorial design to analyze how the model responds to protected attributes (including gender, age, and nationality) across five varying stimulus volumes ranging from K = 16 to 12,000 curricula vitae (CVs). The findings reveal that while the model demonstrates acute sensitivity to objective professional suitability demographic bias does not operate as a static constant. Instead, bias manifests non-linearly as a context-dependent interaction effect. While the model exhibited robust neutrality at extreme low and high stimulus volumes (K = 16 and K = 12,000) significant gender and age-related structural breakdowns occurred at mid-range context densities (K = 3,000 and K = 6,000). In contrast, evaluations regarding nationality remained robustly equitable across all stimulus volumes. Furthermore, a multi-account audit across N= 11 independent user sessions confirms that while stochastic noise significantly shifts absolute scoring for candidate filtering baselines, the underlying discriminatory configurations remain structurally invariant (r > 0.99) even at the smallest sample sizes. These results suggest that absolute score thresholds for candidate filtering are highly unreliable and that latent biases can trigger unpredictably as context windows approach saturation limits. The study concludes with practical implications for digital governance and the necessity of high-volume algorithmic stress tests to ensure non-discriminatory HR selection under the regulatory frameworks of the EU AI Act.

Cite this Paper

BibTeX
@InProceedings{pmlr-v350-grenzebach26a, title = {Probing Protected Attributes within General-Purpose LLMs for Hiring}, author = {Grenzebach, Jan and Rad\"untz, Thea}, booktitle = {Proceedings of Fifth European Conference on Algorithmic Fairness}, pages = {372--385}, year = {2026}, editor = {De Bie, Tijl and Defrance, MaryBeth and Calders, Toon and Nguyen, Dennis}, volume = {350}, series = {Proceedings of Machine Learning Research}, month = {02--04 Sep}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v350/main/assets/grenzebach26a/grenzebach26a.pdf}, url = {https://proceedings.mlr.press/v350/grenzebach26a.html}, abstract = {General-purpose Large Language Models (LLM) applications in human resource workflows are primarily driven by the need for objectivity and efficiency. However, a significant gap exists between enterprise-level AI and the consumer-grade ”black-box” services frequently utilized by small and medium-sized enterprises (SMEs), raising critical questions regarding algorithmic consistency and fairness. This paper utilizes the IN-OUT framework, a stimulus-response methodology derived from the cognitive sciences, to empirically audit the behavior of one popular LLM. We employ a 2 $\times$ 2 factorial design to analyze how the model responds to protected attributes (including gender, age, and nationality) across five varying stimulus volumes ranging from K = 16 to 12,000 curricula vitae (CVs). The findings reveal that while the model demonstrates acute sensitivity to objective professional suitability demographic bias does not operate as a static constant. Instead, bias manifests non-linearly as a context-dependent interaction effect. While the model exhibited robust neutrality at extreme low and high stimulus volumes (K = 16 and K = 12,000) significant gender and age-related structural breakdowns occurred at mid-range context densities (K = 3,000 and K = 6,000). In contrast, evaluations regarding nationality remained robustly equitable across all stimulus volumes. Furthermore, a multi-account audit across N= 11 independent user sessions confirms that while stochastic noise significantly shifts absolute scoring for candidate filtering baselines, the underlying discriminatory configurations remain structurally invariant (r > 0.99) even at the smallest sample sizes. These results suggest that absolute score thresholds for candidate filtering are highly unreliable and that latent biases can trigger unpredictably as context windows approach saturation limits. The study concludes with practical implications for digital governance and the necessity of high-volume algorithmic stress tests to ensure non-discriminatory HR selection under the regulatory frameworks of the EU AI Act.} }
Endnote
%0 Conference Paper %T Probing Protected Attributes within General-Purpose LLMs for Hiring %A Jan Grenzebach %A Thea Radüntz %B Proceedings of Fifth European Conference on Algorithmic Fairness %C Proceedings of Machine Learning Research %D 2026 %E Tijl De Bie %E MaryBeth Defrance %E Toon Calders %E Dennis Nguyen %F pmlr-v350-grenzebach26a %I PMLR %P 372--385 %U https://proceedings.mlr.press/v350/grenzebach26a.html %V 350 %X General-purpose Large Language Models (LLM) applications in human resource workflows are primarily driven by the need for objectivity and efficiency. However, a significant gap exists between enterprise-level AI and the consumer-grade ”black-box” services frequently utilized by small and medium-sized enterprises (SMEs), raising critical questions regarding algorithmic consistency and fairness. This paper utilizes the IN-OUT framework, a stimulus-response methodology derived from the cognitive sciences, to empirically audit the behavior of one popular LLM. We employ a 2 $\times$ 2 factorial design to analyze how the model responds to protected attributes (including gender, age, and nationality) across five varying stimulus volumes ranging from K = 16 to 12,000 curricula vitae (CVs). The findings reveal that while the model demonstrates acute sensitivity to objective professional suitability demographic bias does not operate as a static constant. Instead, bias manifests non-linearly as a context-dependent interaction effect. While the model exhibited robust neutrality at extreme low and high stimulus volumes (K = 16 and K = 12,000) significant gender and age-related structural breakdowns occurred at mid-range context densities (K = 3,000 and K = 6,000). In contrast, evaluations regarding nationality remained robustly equitable across all stimulus volumes. Furthermore, a multi-account audit across N= 11 independent user sessions confirms that while stochastic noise significantly shifts absolute scoring for candidate filtering baselines, the underlying discriminatory configurations remain structurally invariant (r > 0.99) even at the smallest sample sizes. These results suggest that absolute score thresholds for candidate filtering are highly unreliable and that latent biases can trigger unpredictably as context windows approach saturation limits. The study concludes with practical implications for digital governance and the necessity of high-volume algorithmic stress tests to ensure non-discriminatory HR selection under the regulatory frameworks of the EU AI Act.
APA
Grenzebach, J. & Radüntz, T.. (2026). Probing Protected Attributes within General-Purpose LLMs for Hiring. Proceedings of Fifth European Conference on Algorithmic Fairness, in Proceedings of Machine Learning Research 350:372-385 Available from https://proceedings.mlr.press/v350/grenzebach26a.html.

Related Material