Dynamic Coordination Graph for Cooperative Multi-Agent Reinforcement Learning

Chapman Siu, Jason Traish, Richard Yi Da Xu
Proceedings of The 13th Asian Conference on Machine Learning, PMLR 157:438-453, 2021.

Abstract

This paper introduces Dynamic $Q$-value Coordination Graph (QCGraph) for cooperative multi-agent reinforcement learning. QCGraph aims to dynamically represent and generalize through factorizing the joint value function of all agents according to dynamically created coordination graph based on subsets of agents. The value can be maximized by message passing at both a local and global level along the graph which allows training the value function end-to-end. The coordination graph is dynamically generated and used to generate the payoff functions which are approximated using graph neural networks and parameter sharing to improve generalization over the state-action space. We show that QCGraph can solve a variety of challenging multi-agent tasks being superior to other value factorization approaches.

Cite this Paper

BibTeX
@InProceedings{pmlr-v157-siu21a, title = {Dynamic Coordination Graph for Cooperative Multi-Agent Reinforcement Learning}, author = {Siu, Chapman and Traish, Jason and Xu, Richard Yi Da}, booktitle = {Proceedings of The 13th Asian Conference on Machine Learning}, pages = {438--453}, year = {2021}, editor = {Balasubramanian, Vineeth N. and Tsang, Ivor}, volume = {157}, series = {Proceedings of Machine Learning Research}, month = {17--19 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v157/siu21a/siu21a.pdf}, url = {https://proceedings.mlr.press/v157/siu21a.html}, abstract = {This paper introduces Dynamic $Q$-value Coordination Graph (QCGraph) for cooperative multi-agent reinforcement learning. QCGraph aims to dynamically represent and generalize through factorizing the joint value function of all agents according to dynamically created coordination graph based on subsets of agents. The value can be maximized by message passing at both a local and global level along the graph which allows training the value function end-to-end. The coordination graph is dynamically generated and used to generate the payoff functions which are approximated using graph neural networks and parameter sharing to improve generalization over the state-action space. We show that QCGraph can solve a variety of challenging multi-agent tasks being superior to other value factorization approaches. } }
Endnote
%0 Conference Paper %T Dynamic Coordination Graph for Cooperative Multi-Agent Reinforcement Learning %A Chapman Siu %A Jason Traish %A Richard Yi Da Xu %B Proceedings of The 13th Asian Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Vineeth N. Balasubramanian %E Ivor Tsang %F pmlr-v157-siu21a %I PMLR %P 438--453 %U https://proceedings.mlr.press/v157/siu21a.html %V 157 %X This paper introduces Dynamic $Q$-value Coordination Graph (QCGraph) for cooperative multi-agent reinforcement learning. QCGraph aims to dynamically represent and generalize through factorizing the joint value function of all agents according to dynamically created coordination graph based on subsets of agents. The value can be maximized by message passing at both a local and global level along the graph which allows training the value function end-to-end. The coordination graph is dynamically generated and used to generate the payoff functions which are approximated using graph neural networks and parameter sharing to improve generalization over the state-action space. We show that QCGraph can solve a variety of challenging multi-agent tasks being superior to other value factorization approaches.
APA
Siu, C., Traish, J. & Xu, R.Y.D.. (2021). Dynamic Coordination Graph for Cooperative Multi-Agent Reinforcement Learning. Proceedings of The 13th Asian Conference on Machine Learning, in Proceedings of Machine Learning Research 157:438-453 Available from https://proceedings.mlr.press/v157/siu21a.html.

Related Material