Fair Representation Learning through Implicit Path Alignment

Changjian Shui, Qi Chen, Jiaqi Li, Boyu Wang, Christian Gagné
Proceedings of the 39th International Conference on Machine Learning, PMLR 162:20156-20175, 2022.

Abstract

We consider a fair representation learning perspective, where optimal predictors, on top of the data representation, are ensured to be invariant with respect to different sub-groups. Specifically, we formulate this intuition as a bi-level optimization, where the representation is learned in the outer-loop, and invariant optimal group predictors are updated in the inner-loop. Moreover, the proposed bi-level objective is demonstrated to fulfill the sufficiency rule, which is desirable in various practical scenarios but was not commonly studied in the fair learning. Besides, to avoid the high computational and memory cost of differentiating in the inner-loop of bi-level objective, we propose an implicit path alignment algorithm, which only relies on the solution of inner optimization and the implicit differentiation rather than the exact optimization path. We further analyze the error gap of the implicit approach and empirically validate the proposed method in both classification and regression settings. Experimental results show the consistently better trade-off in prediction performance and fairness measurement.

Cite this Paper

BibTeX
@InProceedings{pmlr-v162-shui22a, title = {Fair Representation Learning through Implicit Path Alignment}, author = {Shui, Changjian and Chen, Qi and Li, Jiaqi and Wang, Boyu and Gagn{\'e}, Christian}, booktitle = {Proceedings of the 39th International Conference on Machine Learning}, pages = {20156--20175}, year = {2022}, editor = {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan}, volume = {162}, series = {Proceedings of Machine Learning Research}, month = {17--23 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v162/shui22a/shui22a.pdf}, url = {https://proceedings.mlr.press/v162/shui22a.html}, abstract = {We consider a fair representation learning perspective, where optimal predictors, on top of the data representation, are ensured to be invariant with respect to different sub-groups. Specifically, we formulate this intuition as a bi-level optimization, where the representation is learned in the outer-loop, and invariant optimal group predictors are updated in the inner-loop. Moreover, the proposed bi-level objective is demonstrated to fulfill the sufficiency rule, which is desirable in various practical scenarios but was not commonly studied in the fair learning. Besides, to avoid the high computational and memory cost of differentiating in the inner-loop of bi-level objective, we propose an implicit path alignment algorithm, which only relies on the solution of inner optimization and the implicit differentiation rather than the exact optimization path. We further analyze the error gap of the implicit approach and empirically validate the proposed method in both classification and regression settings. Experimental results show the consistently better trade-off in prediction performance and fairness measurement.} }
Endnote
%0 Conference Paper %T Fair Representation Learning through Implicit Path Alignment %A Changjian Shui %A Qi Chen %A Jiaqi Li %A Boyu Wang %A Christian Gagné %B Proceedings of the 39th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2022 %E Kamalika Chaudhuri %E Stefanie Jegelka %E Le Song %E Csaba Szepesvari %E Gang Niu %E Sivan Sabato %F pmlr-v162-shui22a %I PMLR %P 20156--20175 %U https://proceedings.mlr.press/v162/shui22a.html %V 162 %X We consider a fair representation learning perspective, where optimal predictors, on top of the data representation, are ensured to be invariant with respect to different sub-groups. Specifically, we formulate this intuition as a bi-level optimization, where the representation is learned in the outer-loop, and invariant optimal group predictors are updated in the inner-loop. Moreover, the proposed bi-level objective is demonstrated to fulfill the sufficiency rule, which is desirable in various practical scenarios but was not commonly studied in the fair learning. Besides, to avoid the high computational and memory cost of differentiating in the inner-loop of bi-level objective, we propose an implicit path alignment algorithm, which only relies on the solution of inner optimization and the implicit differentiation rather than the exact optimization path. We further analyze the error gap of the implicit approach and empirically validate the proposed method in both classification and regression settings. Experimental results show the consistently better trade-off in prediction performance and fairness measurement.
APA
Shui, C., Chen, Q., Li, J., Wang, B. & Gagné, C.. (2022). Fair Representation Learning through Implicit Path Alignment. Proceedings of the 39th International Conference on Machine Learning, in Proceedings of Machine Learning Research 162:20156-20175 Available from https://proceedings.mlr.press/v162/shui22a.html.

Related Material