Learning Policy Representations in Multiagent Systems

Aditya Grover, Maruan Al-Shedivat, Jayesh Gupta, Yuri Burda, Harrison Edwards
Proceedings of the 35th International Conference on Machine Learning, PMLR 80:1802-1811, 2018.

Abstract

Modeling agent behavior is central to understanding the emergence of complex phenomena in multiagent systems. Prior work in agent modeling has largely been task-specific and driven by hand-engineering domain-specific prior knowledge. We propose a general learning framework for modeling agent behavior in any multiagent system using only a handful of interaction data. Our framework casts agent modeling as a representation learning problem. Consequently, we construct a novel objective inspired by imitation learning and agent identification and design an algorithm for unsupervised learning of representations of agent policies. We demonstrate empirically the utility of the proposed framework in (i) a challenging high-dimensional competitive environment for continuous control and (ii) a cooperative environment for communication, on supervised predictive tasks, unsupervised clustering, and policy optimization using deep reinforcement learning.

Cite this Paper


BibTeX
@InProceedings{pmlr-v80-grover18a, title = {Learning Policy Representations in Multiagent Systems}, author = {Grover, Aditya and Al-Shedivat, Maruan and Gupta, Jayesh and Burda, Yuri and Edwards, Harrison}, booktitle = {Proceedings of the 35th International Conference on Machine Learning}, pages = {1802--1811}, year = {2018}, editor = {Dy, Jennifer and Krause, Andreas}, volume = {80}, series = {Proceedings of Machine Learning Research}, month = {10--15 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v80/grover18a/grover18a.pdf}, url = {https://proceedings.mlr.press/v80/grover18a.html}, abstract = {Modeling agent behavior is central to understanding the emergence of complex phenomena in multiagent systems. Prior work in agent modeling has largely been task-specific and driven by hand-engineering domain-specific prior knowledge. We propose a general learning framework for modeling agent behavior in any multiagent system using only a handful of interaction data. Our framework casts agent modeling as a representation learning problem. Consequently, we construct a novel objective inspired by imitation learning and agent identification and design an algorithm for unsupervised learning of representations of agent policies. We demonstrate empirically the utility of the proposed framework in (i) a challenging high-dimensional competitive environment for continuous control and (ii) a cooperative environment for communication, on supervised predictive tasks, unsupervised clustering, and policy optimization using deep reinforcement learning.} }
Endnote
%0 Conference Paper %T Learning Policy Representations in Multiagent Systems %A Aditya Grover %A Maruan Al-Shedivat %A Jayesh Gupta %A Yuri Burda %A Harrison Edwards %B Proceedings of the 35th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2018 %E Jennifer Dy %E Andreas Krause %F pmlr-v80-grover18a %I PMLR %P 1802--1811 %U https://proceedings.mlr.press/v80/grover18a.html %V 80 %X Modeling agent behavior is central to understanding the emergence of complex phenomena in multiagent systems. Prior work in agent modeling has largely been task-specific and driven by hand-engineering domain-specific prior knowledge. We propose a general learning framework for modeling agent behavior in any multiagent system using only a handful of interaction data. Our framework casts agent modeling as a representation learning problem. Consequently, we construct a novel objective inspired by imitation learning and agent identification and design an algorithm for unsupervised learning of representations of agent policies. We demonstrate empirically the utility of the proposed framework in (i) a challenging high-dimensional competitive environment for continuous control and (ii) a cooperative environment for communication, on supervised predictive tasks, unsupervised clustering, and policy optimization using deep reinforcement learning.
APA
Grover, A., Al-Shedivat, M., Gupta, J., Burda, Y. & Edwards, H.. (2018). Learning Policy Representations in Multiagent Systems. Proceedings of the 35th International Conference on Machine Learning, in Proceedings of Machine Learning Research 80:1802-1811 Available from https://proceedings.mlr.press/v80/grover18a.html.

Related Material