Remembering to Be Fair: Non-Markovian Fairness in Sequential Decision Making

Parand A. Alamdari, Toryn Q. Klassen, Elliot Creager, Sheila A. Mcilraith
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:906-920, 2024.

Abstract

Fair decision making has largely been studied with respect to a single decision. Here we investigate the notion of fairness in the context of sequential decision making where multiple stakeholders can be affected by the outcomes of decisions. We observe that fairness often depends on the history of the sequential decision-making process, and in this sense that it is inherently non-Markovian. We further observe that fairness often needs to be assessed at time points within the process, not just at the end of the process. To advance our understanding of this class of fairness problems, we explore the notion of non-Markovian fairness in the context of sequential decision making. We identify properties of non-Markovian fairness, including notions of long-term, anytime, periodic, and bounded fairness. We explore the interplay between non-Markovian fairness and memory and how memory can support construction of fair policies. Finally, we introduce the FairQCM algorithm, which can automatically augment its training data to improve sample efficiency in the synthesis of fair policies via reinforcement learning.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-alamdari24a, title = {Remembering to Be Fair: Non-{M}arkovian Fairness in Sequential Decision Making}, author = {Alamdari, Parand A. and Klassen, Toryn Q. and Creager, Elliot and Mcilraith, Sheila A.}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {906--920}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/alamdari24a/alamdari24a.pdf}, url = {https://proceedings.mlr.press/v235/alamdari24a.html}, abstract = {Fair decision making has largely been studied with respect to a single decision. Here we investigate the notion of fairness in the context of sequential decision making where multiple stakeholders can be affected by the outcomes of decisions. We observe that fairness often depends on the history of the sequential decision-making process, and in this sense that it is inherently non-Markovian. We further observe that fairness often needs to be assessed at time points within the process, not just at the end of the process. To advance our understanding of this class of fairness problems, we explore the notion of non-Markovian fairness in the context of sequential decision making. We identify properties of non-Markovian fairness, including notions of long-term, anytime, periodic, and bounded fairness. We explore the interplay between non-Markovian fairness and memory and how memory can support construction of fair policies. Finally, we introduce the FairQCM algorithm, which can automatically augment its training data to improve sample efficiency in the synthesis of fair policies via reinforcement learning.} }
Endnote
%0 Conference Paper %T Remembering to Be Fair: Non-Markovian Fairness in Sequential Decision Making %A Parand A. Alamdari %A Toryn Q. Klassen %A Elliot Creager %A Sheila A. Mcilraith %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-alamdari24a %I PMLR %P 906--920 %U https://proceedings.mlr.press/v235/alamdari24a.html %V 235 %X Fair decision making has largely been studied with respect to a single decision. Here we investigate the notion of fairness in the context of sequential decision making where multiple stakeholders can be affected by the outcomes of decisions. We observe that fairness often depends on the history of the sequential decision-making process, and in this sense that it is inherently non-Markovian. We further observe that fairness often needs to be assessed at time points within the process, not just at the end of the process. To advance our understanding of this class of fairness problems, we explore the notion of non-Markovian fairness in the context of sequential decision making. We identify properties of non-Markovian fairness, including notions of long-term, anytime, periodic, and bounded fairness. We explore the interplay between non-Markovian fairness and memory and how memory can support construction of fair policies. Finally, we introduce the FairQCM algorithm, which can automatically augment its training data to improve sample efficiency in the synthesis of fair policies via reinforcement learning.
APA
Alamdari, P.A., Klassen, T.Q., Creager, E. & Mcilraith, S.A.. (2024). Remembering to Be Fair: Non-Markovian Fairness in Sequential Decision Making. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:906-920 Available from https://proceedings.mlr.press/v235/alamdari24a.html.

Related Material