Meritocratic Fairness for Cross-Population Selection

Michael Kearns, Aaron Roth, Zhiwei Steven Wu
Proceedings of the 34th International Conference on Machine Learning, PMLR 70:1828-1836, 2017.

Abstract

We consider the problem of selecting a strong pool of individuals from several populations with incomparable skills (e.g. soccer players, mathematicians, and singers) in a fair manner. The quality of an individual is defined to be their relative rank (by cumulative distribution value) within their own population, which permits cross-population comparisons. We study algorithms which attempt to select the highest quality subset despite the fact that true CDF values are not known, and can only be estimated from the finite pool of candidates. Specifically, we quantify the regret in quality imposed by “meritocratic” notions of fairness, which require that individuals are selected with probability that is monotonically increasing in their true quality. We give algorithms with provable fairness and regret guarantees, as well as lower bounds, and provide empirical results which suggest that our algorithms perform better than the theory suggests. We extend our results to a sequential batch setting, in which an algorithm must repeatedly select subsets of individuals from new pools of applicants, but has the benefit of being able to compare them to the accumulated data from previous rounds.

Cite this Paper

BibTeX
@InProceedings{pmlr-v70-kearns17a, title = {Meritocratic Fairness for Cross-Population Selection}, author = {Michael Kearns and Aaron Roth and Zhiwei Steven Wu}, booktitle = {Proceedings of the 34th International Conference on Machine Learning}, pages = {1828--1836}, year = {2017}, editor = {Precup, Doina and Teh, Yee Whye}, volume = {70}, series = {Proceedings of Machine Learning Research}, month = {06--11 Aug}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v70/kearns17a/kearns17a.pdf}, url = {https://proceedings.mlr.press/v70/kearns17a.html}, abstract = {We consider the problem of selecting a strong pool of individuals from several populations with incomparable skills (e.g. soccer players, mathematicians, and singers) in a fair manner. The quality of an individual is defined to be their relative rank (by cumulative distribution value) within their own population, which permits cross-population comparisons. We study algorithms which attempt to select the highest quality subset despite the fact that true CDF values are not known, and can only be estimated from the finite pool of candidates. Specifically, we quantify the regret in quality imposed by “meritocratic” notions of fairness, which require that individuals are selected with probability that is monotonically increasing in their true quality. We give algorithms with provable fairness and regret guarantees, as well as lower bounds, and provide empirical results which suggest that our algorithms perform better than the theory suggests. We extend our results to a sequential batch setting, in which an algorithm must repeatedly select subsets of individuals from new pools of applicants, but has the benefit of being able to compare them to the accumulated data from previous rounds.} }
Endnote
%0 Conference Paper %T Meritocratic Fairness for Cross-Population Selection %A Michael Kearns %A Aaron Roth %A Zhiwei Steven Wu %B Proceedings of the 34th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2017 %E Doina Precup %E Yee Whye Teh %F pmlr-v70-kearns17a %I PMLR %P 1828--1836 %U https://proceedings.mlr.press/v70/kearns17a.html %V 70 %X We consider the problem of selecting a strong pool of individuals from several populations with incomparable skills (e.g. soccer players, mathematicians, and singers) in a fair manner. The quality of an individual is defined to be their relative rank (by cumulative distribution value) within their own population, which permits cross-population comparisons. We study algorithms which attempt to select the highest quality subset despite the fact that true CDF values are not known, and can only be estimated from the finite pool of candidates. Specifically, we quantify the regret in quality imposed by “meritocratic” notions of fairness, which require that individuals are selected with probability that is monotonically increasing in their true quality. We give algorithms with provable fairness and regret guarantees, as well as lower bounds, and provide empirical results which suggest that our algorithms perform better than the theory suggests. We extend our results to a sequential batch setting, in which an algorithm must repeatedly select subsets of individuals from new pools of applicants, but has the benefit of being able to compare them to the accumulated data from previous rounds.
APA
Kearns, M., Roth, A. & Wu, Z.S.. (2017). Meritocratic Fairness for Cross-Population Selection. Proceedings of the 34th International Conference on Machine Learning, in Proceedings of Machine Learning Research 70:1828-1836 Available from https://proceedings.mlr.press/v70/kearns17a.html.

Related Material