Actor-Attention-Critic for Multi-Agent Reinforcement Learning

Shariq Iqbal, Fei Sha
Proceedings of the 36th International Conference on Machine Learning, PMLR 97:2961-2970, 2019.

Abstract

Reinforcement learning in multi-agent scenarios is important for real-world applications but presents challenges beyond those seen in single-agent settings. We present an actor-critic algorithm that trains decentralized policies in multi-agent settings, using centrally computed critics that share an attention mechanism which selects relevant information for each agent at every timestep. This attention mechanism enables more effective and scalable learning in complex multi-agent environments, when compared to recent approaches. Our approach is applicable not only to cooperative settings with shared rewards, but also individualized reward settings, including adversarial settings, as well as settings that do not provide global states, and it makes no assumptions about the action spaces of the agents. As such, it is flexible enough to be applied to most multi-agent learning problems.

Cite this Paper

BibTeX
@InProceedings{pmlr-v97-iqbal19a, title = {Actor-Attention-Critic for Multi-Agent Reinforcement Learning}, author = {Iqbal, Shariq and Sha, Fei}, booktitle = {Proceedings of the 36th International Conference on Machine Learning}, pages = {2961--2970}, year = {2019}, editor = {Chaudhuri, Kamalika and Salakhutdinov, Ruslan}, volume = {97}, series = {Proceedings of Machine Learning Research}, month = {09--15 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v97/iqbal19a/iqbal19a.pdf}, url = {https://proceedings.mlr.press/v97/iqbal19a.html}, abstract = {Reinforcement learning in multi-agent scenarios is important for real-world applications but presents challenges beyond those seen in single-agent settings. We present an actor-critic algorithm that trains decentralized policies in multi-agent settings, using centrally computed critics that share an attention mechanism which selects relevant information for each agent at every timestep. This attention mechanism enables more effective and scalable learning in complex multi-agent environments, when compared to recent approaches. Our approach is applicable not only to cooperative settings with shared rewards, but also individualized reward settings, including adversarial settings, as well as settings that do not provide global states, and it makes no assumptions about the action spaces of the agents. As such, it is flexible enough to be applied to most multi-agent learning problems.} }
Endnote
%0 Conference Paper %T Actor-Attention-Critic for Multi-Agent Reinforcement Learning %A Shariq Iqbal %A Fei Sha %B Proceedings of the 36th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2019 %E Kamalika Chaudhuri %E Ruslan Salakhutdinov %F pmlr-v97-iqbal19a %I PMLR %P 2961--2970 %U https://proceedings.mlr.press/v97/iqbal19a.html %V 97 %X Reinforcement learning in multi-agent scenarios is important for real-world applications but presents challenges beyond those seen in single-agent settings. We present an actor-critic algorithm that trains decentralized policies in multi-agent settings, using centrally computed critics that share an attention mechanism which selects relevant information for each agent at every timestep. This attention mechanism enables more effective and scalable learning in complex multi-agent environments, when compared to recent approaches. Our approach is applicable not only to cooperative settings with shared rewards, but also individualized reward settings, including adversarial settings, as well as settings that do not provide global states, and it makes no assumptions about the action spaces of the agents. As such, it is flexible enough to be applied to most multi-agent learning problems.
APA
Iqbal, S. & Sha, F.. (2019). Actor-Attention-Critic for Multi-Agent Reinforcement Learning. Proceedings of the 36th International Conference on Machine Learning, in Proceedings of Machine Learning Research 97:2961-2970 Available from https://proceedings.mlr.press/v97/iqbal19a.html.

Related Material