Deep Latent Competition: Learning to Race Using Visual Control Policies in Latent Space

Wilko Schwarting, Tim Seyde, Igor Gilitschenski, Lucas Liebenwein, Ryan Sander, Sertac Karaman, Daniela Rus
Proceedings of the 2020 Conference on Robot Learning, PMLR 155:1855-1870, 2021.

Abstract

Learning competitive behaviors in multi-agent settings such as racing requires long-term reasoning about potential adversarial interactions. This paper presents Deep Latent Competition (DLC), a novel reinforcement learning algorithm that learns competitive visual control policies through self-play in imagination. The DLC agent imagines multi-agent interaction sequences in the compact latent space of a learned world model that combines a joint transition function with opponent viewpoint prediction. Imagined self-play reduces costly sample generation in the real world, while the latent representation enables planning to scale gracefully with observation dimensionality. We demonstrate the effectiveness of our algorithm in learning competitive behaviors on a novel multi-agent racing benchmark that requires planning from image observations. Code and videos available at https://sites.google.com/view/deep-latent-competition.

Cite this Paper

BibTeX
@InProceedings{pmlr-v155-schwarting21a, title = {Deep Latent Competition: Learning to Race Using Visual Control Policies in Latent Space}, author = {Schwarting, Wilko and Seyde, Tim and Gilitschenski, Igor and Liebenwein, Lucas and Sander, Ryan and Karaman, Sertac and Rus, Daniela}, booktitle = {Proceedings of the 2020 Conference on Robot Learning}, pages = {1855--1870}, year = {2021}, editor = {Kober, Jens and Ramos, Fabio and Tomlin, Claire}, volume = {155}, series = {Proceedings of Machine Learning Research}, month = {16--18 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v155/schwarting21a/schwarting21a.pdf}, url = {https://proceedings.mlr.press/v155/schwarting21a.html}, abstract = {Learning competitive behaviors in multi-agent settings such as racing requires long-term reasoning about potential adversarial interactions. This paper presents Deep Latent Competition (DLC), a novel reinforcement learning algorithm that learns competitive visual control policies through self-play in imagination. The DLC agent imagines multi-agent interaction sequences in the compact latent space of a learned world model that combines a joint transition function with opponent viewpoint prediction. Imagined self-play reduces costly sample generation in the real world, while the latent representation enables planning to scale gracefully with observation dimensionality. We demonstrate the effectiveness of our algorithm in learning competitive behaviors on a novel multi-agent racing benchmark that requires planning from image observations. Code and videos available at https://sites.google.com/view/deep-latent-competition.} }
Endnote
%0 Conference Paper %T Deep Latent Competition: Learning to Race Using Visual Control Policies in Latent Space %A Wilko Schwarting %A Tim Seyde %A Igor Gilitschenski %A Lucas Liebenwein %A Ryan Sander %A Sertac Karaman %A Daniela Rus %B Proceedings of the 2020 Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2021 %E Jens Kober %E Fabio Ramos %E Claire Tomlin %F pmlr-v155-schwarting21a %I PMLR %P 1855--1870 %U https://proceedings.mlr.press/v155/schwarting21a.html %V 155 %X Learning competitive behaviors in multi-agent settings such as racing requires long-term reasoning about potential adversarial interactions. This paper presents Deep Latent Competition (DLC), a novel reinforcement learning algorithm that learns competitive visual control policies through self-play in imagination. The DLC agent imagines multi-agent interaction sequences in the compact latent space of a learned world model that combines a joint transition function with opponent viewpoint prediction. Imagined self-play reduces costly sample generation in the real world, while the latent representation enables planning to scale gracefully with observation dimensionality. We demonstrate the effectiveness of our algorithm in learning competitive behaviors on a novel multi-agent racing benchmark that requires planning from image observations. Code and videos available at https://sites.google.com/view/deep-latent-competition.
APA
Schwarting, W., Seyde, T., Gilitschenski, I., Liebenwein, L., Sander, R., Karaman, S. & Rus, D.. (2021). Deep Latent Competition: Learning to Race Using Visual Control Policies in Latent Space. Proceedings of the 2020 Conference on Robot Learning, in Proceedings of Machine Learning Research 155:1855-1870 Available from https://proceedings.mlr.press/v155/schwarting21a.html.

Related Material