Equivariant Reinforcement Learning under Partial Observability

Hai Huu Nguyen, Andrea Baisero, David Klee, Dian Wang, Robert Platt, Christopher Amato
Proceedings of The 7th Conference on Robot Learning, PMLR 229:3309-3320, 2023.

Abstract

Incorporating inductive biases is a promising approach for tackling challenging robot learning domains with sample-efficient solutions. This paper identifies partially observable domains where symmetries can be a useful inductive bias for efficient learning. Specifically, by encoding the equivariance regarding specific group symmetries into the neural networks, our actor-critic reinforcement learning agents can reuse solutions in the past for related scenarios. Consequently, our equivariant agents outperform non-equivariant approaches significantly in terms of sample efficiency and final performance, demonstrated through experiments on a range of robotic tasks in simulation and real hardware.

Cite this Paper

BibTeX
@InProceedings{pmlr-v229-nguyen23a, title = {Equivariant Reinforcement Learning under Partial Observability}, author = {Nguyen, Hai Huu and Baisero, Andrea and Klee, David and Wang, Dian and Platt, Robert and Amato, Christopher}, booktitle = {Proceedings of The 7th Conference on Robot Learning}, pages = {3309--3320}, year = {2023}, editor = {Tan, Jie and Toussaint, Marc and Darvish, Kourosh}, volume = {229}, series = {Proceedings of Machine Learning Research}, month = {06--09 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v229/nguyen23a/nguyen23a.pdf}, url = {https://proceedings.mlr.press/v229/nguyen23a.html}, abstract = {Incorporating inductive biases is a promising approach for tackling challenging robot learning domains with sample-efficient solutions. This paper identifies partially observable domains where symmetries can be a useful inductive bias for efficient learning. Specifically, by encoding the equivariance regarding specific group symmetries into the neural networks, our actor-critic reinforcement learning agents can reuse solutions in the past for related scenarios. Consequently, our equivariant agents outperform non-equivariant approaches significantly in terms of sample efficiency and final performance, demonstrated through experiments on a range of robotic tasks in simulation and real hardware.} }
Endnote
%0 Conference Paper %T Equivariant Reinforcement Learning under Partial Observability %A Hai Huu Nguyen %A Andrea Baisero %A David Klee %A Dian Wang %A Robert Platt %A Christopher Amato %B Proceedings of The 7th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2023 %E Jie Tan %E Marc Toussaint %E Kourosh Darvish %F pmlr-v229-nguyen23a %I PMLR %P 3309--3320 %U https://proceedings.mlr.press/v229/nguyen23a.html %V 229 %X Incorporating inductive biases is a promising approach for tackling challenging robot learning domains with sample-efficient solutions. This paper identifies partially observable domains where symmetries can be a useful inductive bias for efficient learning. Specifically, by encoding the equivariance regarding specific group symmetries into the neural networks, our actor-critic reinforcement learning agents can reuse solutions in the past for related scenarios. Consequently, our equivariant agents outperform non-equivariant approaches significantly in terms of sample efficiency and final performance, demonstrated through experiments on a range of robotic tasks in simulation and real hardware.
APA
Nguyen, H.H., Baisero, A., Klee, D., Wang, D., Platt, R. & Amato, C.. (2023). Equivariant Reinforcement Learning under Partial Observability. Proceedings of The 7th Conference on Robot Learning, in Proceedings of Machine Learning Research 229:3309-3320 Available from https://proceedings.mlr.press/v229/nguyen23a.html.

Related Material