Corruption-Robust Offline Two-Player Zero-Sum Markov Games

Andi Nika, Debmalya Mandal, Adish Singla, Goran Radanovic
Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, PMLR 238:1243-1251, 2024.

Abstract

We study data corruption robustness in offline two-player zero-sum Markov games. Given a dataset of realized trajectories of two players, an adversary is allowed to modify an $\epsilon$-fraction of it. The learner’s goal is to identify an approximate Nash Equilibrium policy pair from the corrupted data. We consider this problem in linear Markov games under different degrees of data coverage and corruption. We start by providing an information-theoretic lower bound on the suboptimality gap of any learner. Next, we propose robust versions of the Pessimistic Minimax Value Iteration algorithm (Zhong et al., 2022), both under coverage on the corrupted data and under coverage only on the clean data, and show that they achieve (near)-optimal suboptimality gap bounds with respect to $\epsilon$. We note that we are the first to provide such a characterization of the problem of learning approximate Nash Equilibrium policies in offline two-player zero-sum Markov games under data corruption.

Cite this Paper

BibTeX
@InProceedings{pmlr-v238-nika24a, title = {Corruption-Robust Offline Two-Player Zero-Sum {M}arkov Games}, author = {Nika, Andi and Mandal, Debmalya and Singla, Adish and Radanovic, Goran}, booktitle = {Proceedings of The 27th International Conference on Artificial Intelligence and Statistics}, pages = {1243--1251}, year = {2024}, editor = {Dasgupta, Sanjoy and Mandt, Stephan and Li, Yingzhen}, volume = {238}, series = {Proceedings of Machine Learning Research}, month = {02--04 May}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v238/nika24a/nika24a.pdf}, url = {https://proceedings.mlr.press/v238/nika24a.html}, abstract = {We study data corruption robustness in offline two-player zero-sum Markov games. Given a dataset of realized trajectories of two players, an adversary is allowed to modify an $\epsilon$-fraction of it. The learner’s goal is to identify an approximate Nash Equilibrium policy pair from the corrupted data. We consider this problem in linear Markov games under different degrees of data coverage and corruption. We start by providing an information-theoretic lower bound on the suboptimality gap of any learner. Next, we propose robust versions of the Pessimistic Minimax Value Iteration algorithm (Zhong et al., 2022), both under coverage on the corrupted data and under coverage only on the clean data, and show that they achieve (near)-optimal suboptimality gap bounds with respect to $\epsilon$. We note that we are the first to provide such a characterization of the problem of learning approximate Nash Equilibrium policies in offline two-player zero-sum Markov games under data corruption.} }
Endnote
%0 Conference Paper %T Corruption-Robust Offline Two-Player Zero-Sum Markov Games %A Andi Nika %A Debmalya Mandal %A Adish Singla %A Goran Radanovic %B Proceedings of The 27th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2024 %E Sanjoy Dasgupta %E Stephan Mandt %E Yingzhen Li %F pmlr-v238-nika24a %I PMLR %P 1243--1251 %U https://proceedings.mlr.press/v238/nika24a.html %V 238 %X We study data corruption robustness in offline two-player zero-sum Markov games. Given a dataset of realized trajectories of two players, an adversary is allowed to modify an $\epsilon$-fraction of it. The learner’s goal is to identify an approximate Nash Equilibrium policy pair from the corrupted data. We consider this problem in linear Markov games under different degrees of data coverage and corruption. We start by providing an information-theoretic lower bound on the suboptimality gap of any learner. Next, we propose robust versions of the Pessimistic Minimax Value Iteration algorithm (Zhong et al., 2022), both under coverage on the corrupted data and under coverage only on the clean data, and show that they achieve (near)-optimal suboptimality gap bounds with respect to $\epsilon$. We note that we are the first to provide such a characterization of the problem of learning approximate Nash Equilibrium policies in offline two-player zero-sum Markov games under data corruption.
APA
Nika, A., Mandal, D., Singla, A. & Radanovic, G.. (2024). Corruption-Robust Offline Two-Player Zero-Sum Markov Games. Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 238:1243-1251 Available from https://proceedings.mlr.press/v238/nika24a.html.

Related Material