Searching for Fairer Machine Learning Ensembles

Michael Feffer, Martin Hirzel, Samuel C Hoffman, Kiran Kate, Parikshit Ram, Avraham Shinnar
Proceedings of the Second International Conference on Automated Machine Learning, PMLR 224:17/1-19, 2023.

Abstract

Bias mitigators can improve algorithmic fairness in machine learning models, but their effect on fairness is often not stable across data splits. A popular approach to train more stable models is ensemble learning, but unfortunately, it is unclear how to combine ensembles with mitigators to best navigate trade-offs between fairness and predictive performance. To that end, we extended the open-source library Lale to enable the modular composition of 8 mitigators, 4 ensembles, and their corresponding hyperparameters, and we empirically explored the space of configurations on 13 datasets. We distilled our insights from this exploration in the form of a guidance diagram that can serve as a starting point for practitioners that we demonstrate is robust and reproducible. We also ran automatic combined algorithm selection and hyperparmeter tuning (or CASH) over ensembles with mitigators. The solutions from the guidance diagram perform similar to those from CASH on many datasets.

Cite this Paper

BibTeX
@InProceedings{pmlr-v224-feffer23a, title = {Searching for Fairer Machine Learning Ensembles}, author = {Feffer, Michael and Hirzel, Martin and Hoffman, Samuel C and Kate, Kiran and Ram, Parikshit and Shinnar, Avraham}, booktitle = {Proceedings of the Second International Conference on Automated Machine Learning}, pages = {17/1--19}, year = {2023}, editor = {Faust, Aleksandra and Garnett, Roman and White, Colin and Hutter, Frank and Gardner, Jacob R.}, volume = {224}, series = {Proceedings of Machine Learning Research}, month = {12--15 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v224/feffer23a/feffer23a.pdf}, url = {https://proceedings.mlr.press/v224/feffer23a.html}, abstract = {Bias mitigators can improve algorithmic fairness in machine learning models, but their effect on fairness is often not stable across data splits. A popular approach to train more stable models is ensemble learning, but unfortunately, it is unclear how to combine ensembles with mitigators to best navigate trade-offs between fairness and predictive performance. To that end, we extended the open-source library Lale to enable the modular composition of 8 mitigators, 4 ensembles, and their corresponding hyperparameters, and we empirically explored the space of configurations on 13 datasets. We distilled our insights from this exploration in the form of a guidance diagram that can serve as a starting point for practitioners that we demonstrate is robust and reproducible. We also ran automatic combined algorithm selection and hyperparmeter tuning (or CASH) over ensembles with mitigators. The solutions from the guidance diagram perform similar to those from CASH on many datasets.} }
Endnote
%0 Conference Paper %T Searching for Fairer Machine Learning Ensembles %A Michael Feffer %A Martin Hirzel %A Samuel C Hoffman %A Kiran Kate %A Parikshit Ram %A Avraham Shinnar %B Proceedings of the Second International Conference on Automated Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Aleksandra Faust %E Roman Garnett %E Colin White %E Frank Hutter %E Jacob R. Gardner %F pmlr-v224-feffer23a %I PMLR %P 17/1--19 %U https://proceedings.mlr.press/v224/feffer23a.html %V 224 %X Bias mitigators can improve algorithmic fairness in machine learning models, but their effect on fairness is often not stable across data splits. A popular approach to train more stable models is ensemble learning, but unfortunately, it is unclear how to combine ensembles with mitigators to best navigate trade-offs between fairness and predictive performance. To that end, we extended the open-source library Lale to enable the modular composition of 8 mitigators, 4 ensembles, and their corresponding hyperparameters, and we empirically explored the space of configurations on 13 datasets. We distilled our insights from this exploration in the form of a guidance diagram that can serve as a starting point for practitioners that we demonstrate is robust and reproducible. We also ran automatic combined algorithm selection and hyperparmeter tuning (or CASH) over ensembles with mitigators. The solutions from the guidance diagram perform similar to those from CASH on many datasets.
APA
Feffer, M., Hirzel, M., Hoffman, S.C., Kate, K., Ram, P. & Shinnar, A.. (2023). Searching for Fairer Machine Learning Ensembles. Proceedings of the Second International Conference on Automated Machine Learning, in Proceedings of Machine Learning Research 224:17/1-19 Available from https://proceedings.mlr.press/v224/feffer23a.html.

Related Material