Selection, Ignorability and Challenges With Causal Fairness

Jake Fawkes, Robin Evans, Dino Sejdinovic
Proceedings of the First Conference on Causal Learning and Reasoning, PMLR 177:275-289, 2022.

Abstract

In this paper we look at popular fairness methods that use causal counterfactuals. These methods capture the intuitive notion that a prediction is fair if it coincides with the prediction that would have been made if someone’s race, gender or religion were counterfactually different. In order to achieve this, we must have causal models that are able to capture what someone would be like if we were to counterfactually change these traits. However, we argue that any model that can do this must lie outside the particularly well behaved class that is commonly considered in the fairness literature. This is because in fairness settings, models in this class entail a particularly strong causal assumption, normally only seen in a randomised controlled trial. We argue that in general this is unlikely to hold. Furthermore, we show in many cases it can be explicitly rejected due to the fact that samples are selected from a wider population. We show this creates difficulties for counterfactual fairness as well as for the application of more general causal fairness methods.

Cite this Paper

BibTeX
@InProceedings{pmlr-v177-fawkes22a, title = {Selection, Ignorability and Challenges With Causal Fairness }, author = {Fawkes, Jake and Evans, Robin and Sejdinovic, Dino}, booktitle = {Proceedings of the First Conference on Causal Learning and Reasoning}, pages = {275--289}, year = {2022}, editor = {Schölkopf, Bernhard and Uhler, Caroline and Zhang, Kun}, volume = {177}, series = {Proceedings of Machine Learning Research}, month = {11--13 Apr}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v177/fawkes22a/fawkes22a.pdf}, url = {https://proceedings.mlr.press/v177/fawkes22a.html}, abstract = {In this paper we look at popular fairness methods that use causal counterfactuals. These methods capture the intuitive notion that a prediction is fair if it coincides with the prediction that would have been made if someone’s race, gender or religion were counterfactually different. In order to achieve this, we must have causal models that are able to capture what someone would be like if we were to counterfactually change these traits. However, we argue that any model that can do this must lie outside the particularly well behaved class that is commonly considered in the fairness literature. This is because in fairness settings, models in this class entail a particularly strong causal assumption, normally only seen in a randomised controlled trial. We argue that in general this is unlikely to hold. Furthermore, we show in many cases it can be explicitly rejected due to the fact that samples are selected from a wider population. We show this creates difficulties for counterfactual fairness as well as for the application of more general causal fairness methods.} }
Endnote
%0 Conference Paper %T Selection, Ignorability and Challenges With Causal Fairness %A Jake Fawkes %A Robin Evans %A Dino Sejdinovic %B Proceedings of the First Conference on Causal Learning and Reasoning %C Proceedings of Machine Learning Research %D 2022 %E Bernhard Schölkopf %E Caroline Uhler %E Kun Zhang %F pmlr-v177-fawkes22a %I PMLR %P 275--289 %U https://proceedings.mlr.press/v177/fawkes22a.html %V 177 %X In this paper we look at popular fairness methods that use causal counterfactuals. These methods capture the intuitive notion that a prediction is fair if it coincides with the prediction that would have been made if someone’s race, gender or religion were counterfactually different. In order to achieve this, we must have causal models that are able to capture what someone would be like if we were to counterfactually change these traits. However, we argue that any model that can do this must lie outside the particularly well behaved class that is commonly considered in the fairness literature. This is because in fairness settings, models in this class entail a particularly strong causal assumption, normally only seen in a randomised controlled trial. We argue that in general this is unlikely to hold. Furthermore, we show in many cases it can be explicitly rejected due to the fact that samples are selected from a wider population. We show this creates difficulties for counterfactual fairness as well as for the application of more general causal fairness methods.
APA
Fawkes, J., Evans, R. & Sejdinovic, D.. (2022). Selection, Ignorability and Challenges With Causal Fairness . Proceedings of the First Conference on Causal Learning and Reasoning, in Proceedings of Machine Learning Research 177:275-289 Available from https://proceedings.mlr.press/v177/fawkes22a.html.

Related Material