Multicalibration: Calibration for the (Computationally-Identifiable) Masses

Ursula Hebert-Johnson, Michael Kim, Omer Reingold, Guy Rothblum
Proceedings of the 35th International Conference on Machine Learning, PMLR 80:1939-1948, 2018.

Abstract

We develop and study multicalibration as a new measure of fairness in machine learning that aims to mitigate inadvertent or malicious discrimination that is introduced at training time (even from ground truth data). Multicalibration guarantees meaningful (calibrated) predictions for every subpopulation that can be identified within a specified class of computations. The specified class can be quite rich; in particular, it can contain many overlapping subgroups of a protected group. We demonstrate that in many settings this strong notion of protection from discrimination is provably attainable and aligned with the goal of obtaining accurate predictions. Along the way, we present algorithms for learning a multicalibrated predictor, study the computational complexity of this task, and illustrate tight connections to the agnostic learning model.

Cite this Paper

BibTeX
@InProceedings{pmlr-v80-hebert-johnson18a, title = {Multicalibration: Calibration for the ({C}omputationally-Identifiable) Masses}, author = {Hebert-Johnson, Ursula and Kim, Michael and Reingold, Omer and Rothblum, Guy}, booktitle = {Proceedings of the 35th International Conference on Machine Learning}, pages = {1939--1948}, year = {2018}, editor = {Dy, Jennifer and Krause, Andreas}, volume = {80}, series = {Proceedings of Machine Learning Research}, month = {10--15 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v80/hebert-johnson18a/hebert-johnson18a.pdf}, url = {https://proceedings.mlr.press/v80/hebert-johnson18a.html}, abstract = {We develop and study multicalibration as a new measure of fairness in machine learning that aims to mitigate inadvertent or malicious discrimination that is introduced at training time (even from ground truth data). Multicalibration guarantees meaningful (calibrated) predictions for every subpopulation that can be identified within a specified class of computations. The specified class can be quite rich; in particular, it can contain many overlapping subgroups of a protected group. We demonstrate that in many settings this strong notion of protection from discrimination is provably attainable and aligned with the goal of obtaining accurate predictions. Along the way, we present algorithms for learning a multicalibrated predictor, study the computational complexity of this task, and illustrate tight connections to the agnostic learning model.} }
Endnote
%0 Conference Paper %T Multicalibration: Calibration for the (Computationally-Identifiable) Masses %A Ursula Hebert-Johnson %A Michael Kim %A Omer Reingold %A Guy Rothblum %B Proceedings of the 35th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2018 %E Jennifer Dy %E Andreas Krause %F pmlr-v80-hebert-johnson18a %I PMLR %P 1939--1948 %U https://proceedings.mlr.press/v80/hebert-johnson18a.html %V 80 %X We develop and study multicalibration as a new measure of fairness in machine learning that aims to mitigate inadvertent or malicious discrimination that is introduced at training time (even from ground truth data). Multicalibration guarantees meaningful (calibrated) predictions for every subpopulation that can be identified within a specified class of computations. The specified class can be quite rich; in particular, it can contain many overlapping subgroups of a protected group. We demonstrate that in many settings this strong notion of protection from discrimination is provably attainable and aligned with the goal of obtaining accurate predictions. Along the way, we present algorithms for learning a multicalibrated predictor, study the computational complexity of this task, and illustrate tight connections to the agnostic learning model.
APA
Hebert-Johnson, U., Kim, M., Reingold, O. & Rothblum, G.. (2018). Multicalibration: Calibration for the (Computationally-Identifiable) Masses. Proceedings of the 35th International Conference on Machine Learning, in Proceedings of Machine Learning Research 80:1939-1948 Available from https://proceedings.mlr.press/v80/hebert-johnson18a.html.

Related Material