Lifelong Robotic Reinforcement Learning by Retaining Experiences

Annie Xie, Chelsea Finn
Proceedings of The 1st Conference on Lifelong Learning Agents, PMLR 199:838-855, 2022.

Abstract

Multi-task learning ideally allows embodied agents such as robots to acquire a diverse repertoire of useful skills. However, many multi-task reinforcement learning efforts assume the agent can collect data from all tasks at all times, which can be unrealistic for physical agents that can only attend to one task at a time. Motivated by the practical constraints of physical learning systems, this work studies lifelong learning as a more natural multi-task learning setup. We present an approach that effectively leverages data collected from previous tasks to cumulatively and efficiently grow the robot’s skill-set. In a series of simulated robotic manipulation experiments, our approach requires less than half the samples than learning each task from scratch, while avoiding the impractical round-robin data collection scheme. On a Franka Emika Panda robot arm, our approach incrementally solves ten challenging tasks, including bottle capping and block insertion.

Cite this Paper

BibTeX
@InProceedings{pmlr-v199-xie22a, title = {Lifelong Robotic Reinforcement Learning by Retaining Experiences}, author = {Xie, Annie and Finn, Chelsea}, booktitle = {Proceedings of The 1st Conference on Lifelong Learning Agents}, pages = {838--855}, year = {2022}, editor = {Chandar, Sarath and Pascanu, Razvan and Precup, Doina}, volume = {199}, series = {Proceedings of Machine Learning Research}, month = {22--24 Aug}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v199/xie22a/xie22a.pdf}, url = {https://proceedings.mlr.press/v199/xie22a.html}, abstract = {Multi-task learning ideally allows embodied agents such as robots to acquire a diverse repertoire of useful skills. However, many multi-task reinforcement learning efforts assume the agent can collect data from all tasks at all times, which can be unrealistic for physical agents that can only attend to one task at a time. Motivated by the practical constraints of physical learning systems, this work studies lifelong learning as a more natural multi-task learning setup. We present an approach that effectively leverages data collected from previous tasks to cumulatively and efficiently grow the robot’s skill-set. In a series of simulated robotic manipulation experiments, our approach requires less than half the samples than learning each task from scratch, while avoiding the impractical round-robin data collection scheme. On a Franka Emika Panda robot arm, our approach incrementally solves ten challenging tasks, including bottle capping and block insertion.} }
Endnote
%0 Conference Paper %T Lifelong Robotic Reinforcement Learning by Retaining Experiences %A Annie Xie %A Chelsea Finn %B Proceedings of The 1st Conference on Lifelong Learning Agents %C Proceedings of Machine Learning Research %D 2022 %E Sarath Chandar %E Razvan Pascanu %E Doina Precup %F pmlr-v199-xie22a %I PMLR %P 838--855 %U https://proceedings.mlr.press/v199/xie22a.html %V 199 %X Multi-task learning ideally allows embodied agents such as robots to acquire a diverse repertoire of useful skills. However, many multi-task reinforcement learning efforts assume the agent can collect data from all tasks at all times, which can be unrealistic for physical agents that can only attend to one task at a time. Motivated by the practical constraints of physical learning systems, this work studies lifelong learning as a more natural multi-task learning setup. We present an approach that effectively leverages data collected from previous tasks to cumulatively and efficiently grow the robot’s skill-set. In a series of simulated robotic manipulation experiments, our approach requires less than half the samples than learning each task from scratch, while avoiding the impractical round-robin data collection scheme. On a Franka Emika Panda robot arm, our approach incrementally solves ten challenging tasks, including bottle capping and block insertion.
APA
Xie, A. & Finn, C.. (2022). Lifelong Robotic Reinforcement Learning by Retaining Experiences. Proceedings of The 1st Conference on Lifelong Learning Agents, in Proceedings of Machine Learning Research 199:838-855 Available from https://proceedings.mlr.press/v199/xie22a.html.

Related Material