Balancing Competing Objectives with Noisy Data: Score-Based Classifiers for Welfare-Aware Machine Learning

Esther Rolf, Max Simchowitz, Sarah Dean, Lydia T. Liu, Daniel Bjorkegren, Moritz Hardt, Joshua Blumenstock
Proceedings of the 37th International Conference on Machine Learning, PMLR 119:8158-8168, 2020.

Abstract

While real-world decisions involve many competing objectives, algorithmic decisions are often evaluated with a single objective function. In this paper, we study algorithmic policies which explicitly trade off between a private objective (such as profit) and a public objective (such as social welfare). We analyze a natural class of policies which trace an empirical Pareto frontier based on learned scores, and focus on how such decisions can be made in noisy or data-limited regimes. Our theoretical results characterize the optimal strategies in this class, bound the Pareto errors due to inaccuracies in the scores, and show an equivalence between optimal strategies and a rich class of fairness-constrained profit-maximizing policies. We then present empirical results in two different contexts — online content recommendation and sustainable abalone fisheries — to underscore the generality of our approach to a wide range of practical decisions. Taken together, these results shed light on inherent trade-offs in using machine learning for decisions that impact social welfare.

Cite this Paper

BibTeX
@InProceedings{pmlr-v119-rolf20a, title = {Balancing Competing Objectives with Noisy Data: Score-Based Classifiers for Welfare-Aware Machine Learning}, author = {Rolf, Esther and Simchowitz, Max and Dean, Sarah and Liu, Lydia T. and Bjorkegren, Daniel and Hardt, Moritz and Blumenstock, Joshua}, booktitle = {Proceedings of the 37th International Conference on Machine Learning}, pages = {8158--8168}, year = {2020}, editor = {III, Hal Daumé and Singh, Aarti}, volume = {119}, series = {Proceedings of Machine Learning Research}, month = {13--18 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v119/rolf20a/rolf20a.pdf}, url = {https://proceedings.mlr.press/v119/rolf20a.html}, abstract = {While real-world decisions involve many competing objectives, algorithmic decisions are often evaluated with a single objective function. In this paper, we study algorithmic policies which explicitly trade off between a private objective (such as profit) and a public objective (such as social welfare). We analyze a natural class of policies which trace an empirical Pareto frontier based on learned scores, and focus on how such decisions can be made in noisy or data-limited regimes. Our theoretical results characterize the optimal strategies in this class, bound the Pareto errors due to inaccuracies in the scores, and show an equivalence between optimal strategies and a rich class of fairness-constrained profit-maximizing policies. We then present empirical results in two different contexts — online content recommendation and sustainable abalone fisheries — to underscore the generality of our approach to a wide range of practical decisions. Taken together, these results shed light on inherent trade-offs in using machine learning for decisions that impact social welfare.} }
Endnote
%0 Conference Paper %T Balancing Competing Objectives with Noisy Data: Score-Based Classifiers for Welfare-Aware Machine Learning %A Esther Rolf %A Max Simchowitz %A Sarah Dean %A Lydia T. Liu %A Daniel Bjorkegren %A Moritz Hardt %A Joshua Blumenstock %B Proceedings of the 37th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2020 %E Hal Daumé III %E Aarti Singh %F pmlr-v119-rolf20a %I PMLR %P 8158--8168 %U https://proceedings.mlr.press/v119/rolf20a.html %V 119 %X While real-world decisions involve many competing objectives, algorithmic decisions are often evaluated with a single objective function. In this paper, we study algorithmic policies which explicitly trade off between a private objective (such as profit) and a public objective (such as social welfare). We analyze a natural class of policies which trace an empirical Pareto frontier based on learned scores, and focus on how such decisions can be made in noisy or data-limited regimes. Our theoretical results characterize the optimal strategies in this class, bound the Pareto errors due to inaccuracies in the scores, and show an equivalence between optimal strategies and a rich class of fairness-constrained profit-maximizing policies. We then present empirical results in two different contexts — online content recommendation and sustainable abalone fisheries — to underscore the generality of our approach to a wide range of practical decisions. Taken together, these results shed light on inherent trade-offs in using machine learning for decisions that impact social welfare.
APA
Rolf, E., Simchowitz, M., Dean, S., Liu, L.T., Bjorkegren, D., Hardt, M. & Blumenstock, J.. (2020). Balancing Competing Objectives with Noisy Data: Score-Based Classifiers for Welfare-Aware Machine Learning. Proceedings of the 37th International Conference on Machine Learning, in Proceedings of Machine Learning Research 119:8158-8168 Available from https://proceedings.mlr.press/v119/rolf20a.html.

Related Material