Towards Formalizing the Bias-Personalization Trade-off in Large Language Models

Sofie Goethals, Manon Reusens
Proceedings of Fifth European Conference on Algorithmic Fairness, PMLR 350:178-183, 2026.

Abstract

Large Language Models (LLMs) are increasingly evaluated for fairness by examining whether their responses vary across demographic groups. However, not all response variation is inherently problematic: some differences reflect undesirable bias, whereas others represent appropriate personalization based on task-relevant user information. For instance, demographic cues may be relevant in response to a question about cervical cancer screening, but should not influence answers about pursuing a master’s degree. This distinction remains largely understudied in existing LLM fairness research. In this ongoing work, we introduce novel metrics to analyze both desired personalization and undesired bias in LLM responses. We conduct an initial case study to analyze the current state of this trade-off within LLMs, with respect to the user’s gender. This work serves as a first step toward formalizing the distinction between bias and personalization.

Cite this Paper

BibTeX
@InProceedings{pmlr-v350-goethals26a, title = {Towards Formalizing the Bias-Personalization Trade-off in Large Language Models}, author = {Goethals, Sofie and Reusens, Manon}, booktitle = {Proceedings of Fifth European Conference on Algorithmic Fairness}, pages = {178--183}, year = {2026}, editor = {De Bie, Tijl and Defrance, MaryBeth and Calders, Toon and Nguyen, Dennis}, volume = {350}, series = {Proceedings of Machine Learning Research}, month = {02--04 Sep}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v350/main/assets/goethals26a/goethals26a.pdf}, url = {https://proceedings.mlr.press/v350/goethals26a.html}, abstract = {Large Language Models (LLMs) are increasingly evaluated for fairness by examining whether their responses vary across demographic groups. However, not all response variation is inherently problematic: some differences reflect undesirable bias, whereas others represent appropriate personalization based on task-relevant user information. For instance, demographic cues may be relevant in response to a question about cervical cancer screening, but should not influence answers about pursuing a master’s degree. This distinction remains largely understudied in existing LLM fairness research. In this ongoing work, we introduce novel metrics to analyze both desired personalization and undesired bias in LLM responses. We conduct an initial case study to analyze the current state of this trade-off within LLMs, with respect to the user’s gender. This work serves as a first step toward formalizing the distinction between bias and personalization.} }
Endnote
%0 Conference Paper %T Towards Formalizing the Bias-Personalization Trade-off in Large Language Models %A Sofie Goethals %A Manon Reusens %B Proceedings of Fifth European Conference on Algorithmic Fairness %C Proceedings of Machine Learning Research %D 2026 %E Tijl De Bie %E MaryBeth Defrance %E Toon Calders %E Dennis Nguyen %F pmlr-v350-goethals26a %I PMLR %P 178--183 %U https://proceedings.mlr.press/v350/goethals26a.html %V 350 %X Large Language Models (LLMs) are increasingly evaluated for fairness by examining whether their responses vary across demographic groups. However, not all response variation is inherently problematic: some differences reflect undesirable bias, whereas others represent appropriate personalization based on task-relevant user information. For instance, demographic cues may be relevant in response to a question about cervical cancer screening, but should not influence answers about pursuing a master’s degree. This distinction remains largely understudied in existing LLM fairness research. In this ongoing work, we introduce novel metrics to analyze both desired personalization and undesired bias in LLM responses. We conduct an initial case study to analyze the current state of this trade-off within LLMs, with respect to the user’s gender. This work serves as a first step toward formalizing the distinction between bias and personalization.
APA
Goethals, S. & Reusens, M.. (2026). Towards Formalizing the Bias-Personalization Trade-off in Large Language Models. Proceedings of Fifth European Conference on Algorithmic Fairness, in Proceedings of Machine Learning Research 350:178-183 Available from https://proceedings.mlr.press/v350/goethals26a.html.

Related Material