Training Well-Generalizing Classifiers for Fairness Metrics and Other Data-Dependent Constraints

Andrew Cotter, Maya Gupta, Heinrich Jiang, Nathan Srebro, Karthik Sridharan, Serena Wang, Blake Woodworth, Seungil You
Proceedings of the 36th International Conference on Machine Learning, PMLR 97:1397-1405, 2019.

Abstract

Classifiers can be trained with data-dependent constraints to satisfy fairness goals, reduce churn, achieve a targeted false positive rate, or other policy goals. We study the generalization performance for such constrained optimization problems, in terms of how well the constraints are satisfied at evaluation time, given that they are satisfied at training time. To improve generalization, we frame the problem as a two-player game where one player optimizes the model parameters on a training dataset, and the other player enforces the constraints on an independent validation dataset. We build on recent work in two-player constrained optimization to show that if one uses this two-dataset approach, then constraint generalization can be significantly improved. As we illustrate experimentally, this approach works not only in theory, but also in practice.

Cite this Paper

BibTeX
@InProceedings{pmlr-v97-cotter19b, title = {Training Well-Generalizing Classifiers for Fairness Metrics and Other Data-Dependent Constraints}, author = {Cotter, Andrew and Gupta, Maya and Jiang, Heinrich and Srebro, Nathan and Sridharan, Karthik and Wang, Serena and Woodworth, Blake and You, Seungil}, booktitle = {Proceedings of the 36th International Conference on Machine Learning}, pages = {1397--1405}, year = {2019}, editor = {Chaudhuri, Kamalika and Salakhutdinov, Ruslan}, volume = {97}, series = {Proceedings of Machine Learning Research}, month = {09--15 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v97/cotter19b/cotter19b.pdf}, url = {https://proceedings.mlr.press/v97/cotter19b.html}, abstract = {Classifiers can be trained with data-dependent constraints to satisfy fairness goals, reduce churn, achieve a targeted false positive rate, or other policy goals. We study the generalization performance for such constrained optimization problems, in terms of how well the constraints are satisfied at evaluation time, given that they are satisfied at training time. To improve generalization, we frame the problem as a two-player game where one player optimizes the model parameters on a training dataset, and the other player enforces the constraints on an independent validation dataset. We build on recent work in two-player constrained optimization to show that if one uses this two-dataset approach, then constraint generalization can be significantly improved. As we illustrate experimentally, this approach works not only in theory, but also in practice.} }
Endnote
%0 Conference Paper %T Training Well-Generalizing Classifiers for Fairness Metrics and Other Data-Dependent Constraints %A Andrew Cotter %A Maya Gupta %A Heinrich Jiang %A Nathan Srebro %A Karthik Sridharan %A Serena Wang %A Blake Woodworth %A Seungil You %B Proceedings of the 36th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2019 %E Kamalika Chaudhuri %E Ruslan Salakhutdinov %F pmlr-v97-cotter19b %I PMLR %P 1397--1405 %U https://proceedings.mlr.press/v97/cotter19b.html %V 97 %X Classifiers can be trained with data-dependent constraints to satisfy fairness goals, reduce churn, achieve a targeted false positive rate, or other policy goals. We study the generalization performance for such constrained optimization problems, in terms of how well the constraints are satisfied at evaluation time, given that they are satisfied at training time. To improve generalization, we frame the problem as a two-player game where one player optimizes the model parameters on a training dataset, and the other player enforces the constraints on an independent validation dataset. We build on recent work in two-player constrained optimization to show that if one uses this two-dataset approach, then constraint generalization can be significantly improved. As we illustrate experimentally, this approach works not only in theory, but also in practice.
APA
Cotter, A., Gupta, M., Jiang, H., Srebro, N., Sridharan, K., Wang, S., Woodworth, B. & You, S.. (2019). Training Well-Generalizing Classifiers for Fairness Metrics and Other Data-Dependent Constraints. Proceedings of the 36th International Conference on Machine Learning, in Proceedings of Machine Learning Research 97:1397-1405 Available from https://proceedings.mlr.press/v97/cotter19b.html.

Related Material