Fair Classification with Instance-dependent Label Noise

Songhua Wu, Mingming Gong, Bo Han, Yang Liu, Tongliang Liu
Proceedings of the First Conference on Causal Learning and Reasoning, PMLR 177:927-943, 2022.

Abstract

With the widespread use of machine learning systems in our daily lives, it is important to consider fairness as a basic requirement when designing these systems, especially when the systems make life-changing decisions, e.g., \textit{COMPAS} algorithm helps judges decide whether to release an offender. For another thing, due to the cheap but imperfect data collection methods, such as crowdsourcing and web crawling, label noise is ubiquitous, which unfortunately makes fairness-aware algorithms even more prejudiced than fairness-unaware ones, and thereby harmful. To tackle these problems, we provide general frameworks for learning fair classifiers with \textit{instance-dependent label noise}. For statistical fairness notions, we rewrite the classification risk and the fairness metric in terms of noisy data and thereby build robust classifiers. For the causality-based fairness notion, we exploit the internal causal structure of data to model the label noise and \textit{counterfactual fairness} simultaneously. Experimental results demonstrate the effectiveness of the proposed methods on real-world datasets with controllable synthetic label noise.

Cite this Paper

BibTeX
@InProceedings{pmlr-v177-wu22b, title = {Fair Classification with Instance-dependent Label Noise}, author = {Wu, Songhua and Gong, Mingming and Han, Bo and Liu, Yang and Liu, Tongliang}, booktitle = {Proceedings of the First Conference on Causal Learning and Reasoning}, pages = {927--943}, year = {2022}, editor = {Schölkopf, Bernhard and Uhler, Caroline and Zhang, Kun}, volume = {177}, series = {Proceedings of Machine Learning Research}, month = {11--13 Apr}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v177/wu22b/wu22b.pdf}, url = {https://proceedings.mlr.press/v177/wu22b.html}, abstract = {With the widespread use of machine learning systems in our daily lives, it is important to consider fairness as a basic requirement when designing these systems, especially when the systems make life-changing decisions, e.g., \textit{COMPAS} algorithm helps judges decide whether to release an offender. For another thing, due to the cheap but imperfect data collection methods, such as crowdsourcing and web crawling, label noise is ubiquitous, which unfortunately makes fairness-aware algorithms even more prejudiced than fairness-unaware ones, and thereby harmful. To tackle these problems, we provide general frameworks for learning fair classifiers with \textit{instance-dependent label noise}. For statistical fairness notions, we rewrite the classification risk and the fairness metric in terms of noisy data and thereby build robust classifiers. For the causality-based fairness notion, we exploit the internal causal structure of data to model the label noise and \textit{counterfactual fairness} simultaneously. Experimental results demonstrate the effectiveness of the proposed methods on real-world datasets with controllable synthetic label noise.} }
Endnote
%0 Conference Paper %T Fair Classification with Instance-dependent Label Noise %A Songhua Wu %A Mingming Gong %A Bo Han %A Yang Liu %A Tongliang Liu %B Proceedings of the First Conference on Causal Learning and Reasoning %C Proceedings of Machine Learning Research %D 2022 %E Bernhard Schölkopf %E Caroline Uhler %E Kun Zhang %F pmlr-v177-wu22b %I PMLR %P 927--943 %U https://proceedings.mlr.press/v177/wu22b.html %V 177 %X With the widespread use of machine learning systems in our daily lives, it is important to consider fairness as a basic requirement when designing these systems, especially when the systems make life-changing decisions, e.g., \textit{COMPAS} algorithm helps judges decide whether to release an offender. For another thing, due to the cheap but imperfect data collection methods, such as crowdsourcing and web crawling, label noise is ubiquitous, which unfortunately makes fairness-aware algorithms even more prejudiced than fairness-unaware ones, and thereby harmful. To tackle these problems, we provide general frameworks for learning fair classifiers with \textit{instance-dependent label noise}. For statistical fairness notions, we rewrite the classification risk and the fairness metric in terms of noisy data and thereby build robust classifiers. For the causality-based fairness notion, we exploit the internal causal structure of data to model the label noise and \textit{counterfactual fairness} simultaneously. Experimental results demonstrate the effectiveness of the proposed methods on real-world datasets with controllable synthetic label noise.
APA
Wu, S., Gong, M., Han, B., Liu, Y. & Liu, T.. (2022). Fair Classification with Instance-dependent Label Noise. Proceedings of the First Conference on Causal Learning and Reasoning, in Proceedings of Machine Learning Research 177:927-943 Available from https://proceedings.mlr.press/v177/wu22b.html.

Related Material