Fairness-Enhancing Data Augmentation Methods for Worst-Group Accuracy

Monica Welfert, Nathan Stromberg, Lalitha Sankar
Proceedings of the Algorithmic Fairness Through the Lens of Metrics and Evaluation, PMLR 279:156-172, 2025.

Abstract

Ensuring fair predictions across many distinct subpopulations in the training data canbe prohibitive for large models. Recently, simple linear last layer retraining strategies,in combination with data augmentation methods such as upweighting and downsamplinghave been shown to achieve state-of-the-art performance for worst-group accuracy, whichquantifies accuracy for the least prevalent subpopulation. For linear last layer retraining andthe abovementioned augmentations, we present a comparison of the optimal worst-groupaccuracy when modeling the distribution of the latent representations (input to the last layer)as Gaussian for each subpopulation. Observing that these augmentation techniques relyheavily on well-labeled subpopulations, we present a comparison of the optimal worst-groupaccuracy in the setting of label noise. We verify our results for both synthetic and largepublicly available datasets.

Cite this Paper

BibTeX
@InProceedings{pmlr-v279-welfert25a, title = {Fairness-Enhancing Data Augmentation Methods for Worst-Group Accuracy}, author = {Welfert, Monica and Stromberg, Nathan and Sankar, Lalitha}, booktitle = {Proceedings of the Algorithmic Fairness Through the Lens of Metrics and Evaluation}, pages = {156--172}, year = {2025}, editor = {Rateike, Miriam and Dieng, Awa and Watson-Daniels, Jamelle and Fioretto, Ferdinando and Farnadi, Golnoosh}, volume = {279}, series = {Proceedings of Machine Learning Research}, month = {14 Dec}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v279/main/assets/welfert25a/welfert25a.pdf}, url = {https://proceedings.mlr.press/v279/welfert25a.html}, abstract = {Ensuring fair predictions across many distinct subpopulations in the training data canbe prohibitive for large models. Recently, simple linear last layer retraining strategies,in combination with data augmentation methods such as upweighting and downsamplinghave been shown to achieve state-of-the-art performance for worst-group accuracy, whichquantifies accuracy for the least prevalent subpopulation. For linear last layer retraining andthe abovementioned augmentations, we present a comparison of the optimal worst-groupaccuracy when modeling the distribution of the latent representations (input to the last layer)as Gaussian for each subpopulation. Observing that these augmentation techniques relyheavily on well-labeled subpopulations, we present a comparison of the optimal worst-groupaccuracy in the setting of label noise. We verify our results for both synthetic and largepublicly available datasets.} }
Endnote
%0 Conference Paper %T Fairness-Enhancing Data Augmentation Methods for Worst-Group Accuracy %A Monica Welfert %A Nathan Stromberg %A Lalitha Sankar %B Proceedings of the Algorithmic Fairness Through the Lens of Metrics and Evaluation %C Proceedings of Machine Learning Research %D 2025 %E Miriam Rateike %E Awa Dieng %E Jamelle Watson-Daniels %E Ferdinando Fioretto %E Golnoosh Farnadi %F pmlr-v279-welfert25a %I PMLR %P 156--172 %U https://proceedings.mlr.press/v279/welfert25a.html %V 279 %X Ensuring fair predictions across many distinct subpopulations in the training data canbe prohibitive for large models. Recently, simple linear last layer retraining strategies,in combination with data augmentation methods such as upweighting and downsamplinghave been shown to achieve state-of-the-art performance for worst-group accuracy, whichquantifies accuracy for the least prevalent subpopulation. For linear last layer retraining andthe abovementioned augmentations, we present a comparison of the optimal worst-groupaccuracy when modeling the distribution of the latent representations (input to the last layer)as Gaussian for each subpopulation. Observing that these augmentation techniques relyheavily on well-labeled subpopulations, we present a comparison of the optimal worst-groupaccuracy in the setting of label noise. We verify our results for both synthetic and largepublicly available datasets.
APA
Welfert, M., Stromberg, N. & Sankar, L.. (2025). Fairness-Enhancing Data Augmentation Methods for Worst-Group Accuracy. Proceedings of the Algorithmic Fairness Through the Lens of Metrics and Evaluation, in Proceedings of Machine Learning Research 279:156-172 Available from https://proceedings.mlr.press/v279/welfert25a.html.

Related Material