Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observability

Shayegan Omidshafiei, Jason Pazis, Christopher Amato, Jonathan P. How, John Vian
Proceedings of the 34th International Conference on Machine Learning, PMLR 70:2681-2690, 2017.

Abstract

Many real-world tasks involve multiple agents with partial observability and limited communication. Learning is challenging in these settings due to local viewpoints of agents, which perceive the world as non-stationary due to concurrently-exploring teammates. Approaches that learn specialized policies for individual tasks face problems when applied to the real world: not only do agents have to learn and store distinct policies for each task, but in practice identities of tasks are often non-observable, making these approaches inapplicable. This paper formalizes and addresses the problem of multi-task multi-agent reinforcement learning under partial observability. We introduce a decentralized single-task learning approach that is robust to concurrent interactions of teammates, and present an approach for distilling single-task policies into a unified policy that performs well across multiple related tasks, without explicit provision of task identity.

Cite this Paper

BibTeX
@InProceedings{pmlr-v70-omidshafiei17a, title = {Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observability}, author = {Shayegan Omidshafiei and Jason Pazis and Christopher Amato and Jonathan P. How and John Vian}, booktitle = {Proceedings of the 34th International Conference on Machine Learning}, pages = {2681--2690}, year = {2017}, editor = {Precup, Doina and Teh, Yee Whye}, volume = {70}, series = {Proceedings of Machine Learning Research}, month = {06--11 Aug}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v70/omidshafiei17a/omidshafiei17a.pdf}, url = {https://proceedings.mlr.press/v70/omidshafiei17a.html}, abstract = {Many real-world tasks involve multiple agents with partial observability and limited communication. Learning is challenging in these settings due to local viewpoints of agents, which perceive the world as non-stationary due to concurrently-exploring teammates. Approaches that learn specialized policies for individual tasks face problems when applied to the real world: not only do agents have to learn and store distinct policies for each task, but in practice identities of tasks are often non-observable, making these approaches inapplicable. This paper formalizes and addresses the problem of multi-task multi-agent reinforcement learning under partial observability. We introduce a decentralized single-task learning approach that is robust to concurrent interactions of teammates, and present an approach for distilling single-task policies into a unified policy that performs well across multiple related tasks, without explicit provision of task identity.} }
Endnote
%0 Conference Paper %T Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observability %A Shayegan Omidshafiei %A Jason Pazis %A Christopher Amato %A Jonathan P. How %A John Vian %B Proceedings of the 34th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2017 %E Doina Precup %E Yee Whye Teh %F pmlr-v70-omidshafiei17a %I PMLR %P 2681--2690 %U https://proceedings.mlr.press/v70/omidshafiei17a.html %V 70 %X Many real-world tasks involve multiple agents with partial observability and limited communication. Learning is challenging in these settings due to local viewpoints of agents, which perceive the world as non-stationary due to concurrently-exploring teammates. Approaches that learn specialized policies for individual tasks face problems when applied to the real world: not only do agents have to learn and store distinct policies for each task, but in practice identities of tasks are often non-observable, making these approaches inapplicable. This paper formalizes and addresses the problem of multi-task multi-agent reinforcement learning under partial observability. We introduce a decentralized single-task learning approach that is robust to concurrent interactions of teammates, and present an approach for distilling single-task policies into a unified policy that performs well across multiple related tasks, without explicit provision of task identity.
APA
Omidshafiei, S., Pazis, J., Amato, C., How, J.P. & Vian, J.. (2017). Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observability. Proceedings of the 34th International Conference on Machine Learning, in Proceedings of Machine Learning Research 70:2681-2690 Available from https://proceedings.mlr.press/v70/omidshafiei17a.html.

Related Material