Large-Scale Multi-Agent Deep FBSDEs

Tianrong Chen, Ziyi O Wang, Ioannis Exarchos, Evangelos Theodorou
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:1740-1748, 2021.

Abstract

In this paper we present a scalable deep learning framework for finding Markovian Nash Equilibria in multi-agent stochastic games using fictitious play. The motivation is inspired by theoretical analysis of Forward Backward Stochastic Differential Equations and their implementation in a deep learning setting, which is the source of our algorithm’s sample efficiency improvement. By taking advantage of the permutation-invariant property of agents in symmetric games, the scalability and performance is further enhanced significantly. We showcase superior performance of our framework over the state-of-the-art deep fictitious play algorithm on an inter-bank lending/borrowing problem in terms of multiple metrics. More importantly, our approach scales up to 3000 agents in simulation, a scale which, to the best of our knowledge, represents a new state-of-the-art. We also demonstrate the applicability of our framework in robotics on a belief space autonomous racing problem.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-chen21t, title = {Large-Scale Multi-Agent Deep FBSDEs}, author = {Chen, Tianrong and Wang, Ziyi O and Exarchos, Ioannis and Theodorou, Evangelos}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {1740--1748}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/chen21t/chen21t.pdf}, url = {https://proceedings.mlr.press/v139/chen21t.html}, abstract = {In this paper we present a scalable deep learning framework for finding Markovian Nash Equilibria in multi-agent stochastic games using fictitious play. The motivation is inspired by theoretical analysis of Forward Backward Stochastic Differential Equations and their implementation in a deep learning setting, which is the source of our algorithm’s sample efficiency improvement. By taking advantage of the permutation-invariant property of agents in symmetric games, the scalability and performance is further enhanced significantly. We showcase superior performance of our framework over the state-of-the-art deep fictitious play algorithm on an inter-bank lending/borrowing problem in terms of multiple metrics. More importantly, our approach scales up to 3000 agents in simulation, a scale which, to the best of our knowledge, represents a new state-of-the-art. We also demonstrate the applicability of our framework in robotics on a belief space autonomous racing problem.} }
Endnote
%0 Conference Paper %T Large-Scale Multi-Agent Deep FBSDEs %A Tianrong Chen %A Ziyi O Wang %A Ioannis Exarchos %A Evangelos Theodorou %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-chen21t %I PMLR %P 1740--1748 %U https://proceedings.mlr.press/v139/chen21t.html %V 139 %X In this paper we present a scalable deep learning framework for finding Markovian Nash Equilibria in multi-agent stochastic games using fictitious play. The motivation is inspired by theoretical analysis of Forward Backward Stochastic Differential Equations and their implementation in a deep learning setting, which is the source of our algorithm’s sample efficiency improvement. By taking advantage of the permutation-invariant property of agents in symmetric games, the scalability and performance is further enhanced significantly. We showcase superior performance of our framework over the state-of-the-art deep fictitious play algorithm on an inter-bank lending/borrowing problem in terms of multiple metrics. More importantly, our approach scales up to 3000 agents in simulation, a scale which, to the best of our knowledge, represents a new state-of-the-art. We also demonstrate the applicability of our framework in robotics on a belief space autonomous racing problem.
APA
Chen, T., Wang, Z.O., Exarchos, I. & Theodorou, E.. (2021). Large-Scale Multi-Agent Deep FBSDEs. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:1740-1748 Available from https://proceedings.mlr.press/v139/chen21t.html.

Related Material