Privacy Aware Experimentation over Sensitive Groups: A General Chi Square Approach

Rina Friedberg, Ryan Rogers
Proceedings of the Workshop on Algorithmic Fairness through the Lens of Causality and Privacy, PMLR 214:23-66, 2023.

Abstract

As companies work to provide the best possible experience for members, users, and customers, it is crucial to understand how different people – particularly individuals from sensitive groups - have different experiences. For example, do women visit our platform less frequently than members of other genders? Or perhaps, are people with disabilities disproportionately affected by a change to our user interface? However, to run these statistical tests or form estimates to answer these questions, we need to know sensitive attributes. When dealing with personal data, privacy techniques should be considered, especially when we are dealing with sensitive groups, e.g. race/ethnicity or gender. We study a new privacy model where users belong to certain sensitive groups, and we show how to conduct statistical inference on whether there are significant differences in outcomes between the various groups. We introduce a general chi-squared test that accounts for differential privacy in group membership, and show how this covers a broad set of hypothesis tests, improving statistical power over tests that ignore the noise due to privacy.

Cite this Paper

BibTeX
@InProceedings{pmlr-v214-friedberg23a, title = {Privacy Aware Experimentation over Sensitive Groups: A General Chi Square Approach}, author = {Friedberg, Rina and Rogers, Ryan}, booktitle = {Proceedings of the Workshop on Algorithmic Fairness through the Lens of Causality and Privacy}, pages = {23--66}, year = {2023}, editor = {Dieng, Awa and Rateike, Miriam and Farnadi, Golnoosh and Fioretto, Ferdinando and Kusner, Matt and Schrouff, Jessica}, volume = {214}, series = {Proceedings of Machine Learning Research}, month = {03 Dec}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v214/friedberg23a/friedberg23a.pdf}, url = {https://proceedings.mlr.press/v214/friedberg23a.html}, abstract = {As companies work to provide the best possible experience for members, users, and customers, it is crucial to understand how different people – particularly individuals from sensitive groups - have different experiences. For example, do women visit our platform less frequently than members of other genders? Or perhaps, are people with disabilities disproportionately affected by a change to our user interface? However, to run these statistical tests or form estimates to answer these questions, we need to know sensitive attributes. When dealing with personal data, privacy techniques should be considered, especially when we are dealing with sensitive groups, e.g. race/ethnicity or gender. We study a new privacy model where users belong to certain sensitive groups, and we show how to conduct statistical inference on whether there are significant differences in outcomes between the various groups. We introduce a general chi-squared test that accounts for differential privacy in group membership, and show how this covers a broad set of hypothesis tests, improving statistical power over tests that ignore the noise due to privacy.} }
Endnote
%0 Conference Paper %T Privacy Aware Experimentation over Sensitive Groups: A General Chi Square Approach %A Rina Friedberg %A Ryan Rogers %B Proceedings of the Workshop on Algorithmic Fairness through the Lens of Causality and Privacy %C Proceedings of Machine Learning Research %D 2023 %E Awa Dieng %E Miriam Rateike %E Golnoosh Farnadi %E Ferdinando Fioretto %E Matt Kusner %E Jessica Schrouff %F pmlr-v214-friedberg23a %I PMLR %P 23--66 %U https://proceedings.mlr.press/v214/friedberg23a.html %V 214 %X As companies work to provide the best possible experience for members, users, and customers, it is crucial to understand how different people – particularly individuals from sensitive groups - have different experiences. For example, do women visit our platform less frequently than members of other genders? Or perhaps, are people with disabilities disproportionately affected by a change to our user interface? However, to run these statistical tests or form estimates to answer these questions, we need to know sensitive attributes. When dealing with personal data, privacy techniques should be considered, especially when we are dealing with sensitive groups, e.g. race/ethnicity or gender. We study a new privacy model where users belong to certain sensitive groups, and we show how to conduct statistical inference on whether there are significant differences in outcomes between the various groups. We introduce a general chi-squared test that accounts for differential privacy in group membership, and show how this covers a broad set of hypothesis tests, improving statistical power over tests that ignore the noise due to privacy.
APA
Friedberg, R. & Rogers, R.. (2023). Privacy Aware Experimentation over Sensitive Groups: A General Chi Square Approach. Proceedings of the Workshop on Algorithmic Fairness through the Lens of Causality and Privacy, in Proceedings of Machine Learning Research 214:23-66 Available from https://proceedings.mlr.press/v214/friedberg23a.html.

Related Material