Efficient RL via Disentangled Environment and Agent Representations

Kevin Gmelin, Shikhar Bahl, Russell Mendonca, Deepak Pathak
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:11525-11545, 2023.

Abstract

Agents that are aware of the separation between the environments and themselves can leverage this understanding to form effective representations of visual input. We propose an approach for learning such structured representations for RL algorithms, using visual knowledge of the agent, which is often inexpensive to obtain, such as its shape or mask. This is incorporated into the RL objective using a simple auxiliary loss. We show that our method, SEAR (Structured Environment-Agent Representations), outperforms state-of-the-art model-free approaches over 18 different challenging visual simulation environments spanning 5 different robots.

Cite this Paper

BibTeX
@InProceedings{pmlr-v202-gmelin23a, title = {Efficient {RL} via Disentangled Environment and Agent Representations}, author = {Gmelin, Kevin and Bahl, Shikhar and Mendonca, Russell and Pathak, Deepak}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {11525--11545}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/gmelin23a/gmelin23a.pdf}, url = {https://proceedings.mlr.press/v202/gmelin23a.html}, abstract = {Agents that are aware of the separation between the environments and themselves can leverage this understanding to form effective representations of visual input. We propose an approach for learning such structured representations for RL algorithms, using visual knowledge of the agent, which is often inexpensive to obtain, such as its shape or mask. This is incorporated into the RL objective using a simple auxiliary loss. We show that our method, SEAR (Structured Environment-Agent Representations), outperforms state-of-the-art model-free approaches over 18 different challenging visual simulation environments spanning 5 different robots.} }
Endnote
%0 Conference Paper %T Efficient RL via Disentangled Environment and Agent Representations %A Kevin Gmelin %A Shikhar Bahl %A Russell Mendonca %A Deepak Pathak %B Proceedings of the 40th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Andreas Krause %E Emma Brunskill %E Kyunghyun Cho %E Barbara Engelhardt %E Sivan Sabato %E Jonathan Scarlett %F pmlr-v202-gmelin23a %I PMLR %P 11525--11545 %U https://proceedings.mlr.press/v202/gmelin23a.html %V 202 %X Agents that are aware of the separation between the environments and themselves can leverage this understanding to form effective representations of visual input. We propose an approach for learning such structured representations for RL algorithms, using visual knowledge of the agent, which is often inexpensive to obtain, such as its shape or mask. This is incorporated into the RL objective using a simple auxiliary loss. We show that our method, SEAR (Structured Environment-Agent Representations), outperforms state-of-the-art model-free approaches over 18 different challenging visual simulation environments spanning 5 different robots.
APA
Gmelin, K., Bahl, S., Mendonca, R. & Pathak, D.. (2023). Efficient RL via Disentangled Environment and Agent Representations. Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:11525-11545 Available from https://proceedings.mlr.press/v202/gmelin23a.html.

Related Material