Learning Multi-Objective Curricula for Robotic Policy Learning

Jikun Kang, Miao Liu, Abhinav Gupta, Christopher Pal, Xue Liu, Jie Fu
Proceedings of The 6th Conference on Robot Learning, PMLR 205:847-858, 2023.

Abstract

Various automatic curriculum learning (ACL) methods have been proposed to improve the sample efficiency and final performance of robots’ policies learning. They are designed to control how a robotic agent collects data, which is inspired by how humans gradually adapt their learning processes to their capabilities. In this paper, we propose a unified automatic curriculum learning framework to create multi-objective but coherent curricula that are generated by a set of parametric curriculum modules. Each curriculum module is instantiated as a neural network and is responsible for generating a particular curriculum. In order to coordinate those potentially conflicting modules in unified parameter space, we propose a multi-task hyper-net learning framework that uses a single hyper-net to parameterize all those curriculum modules. We evaluate our method on a series of robotic manipulation tasks and demonstrate its superiority over other state-of-the-art ACL methods in terms of sample efficiency and final performance.

Cite this Paper

BibTeX
@InProceedings{pmlr-v205-kang23a, title = {Learning Multi-Objective Curricula for Robotic Policy Learning}, author = {Kang, Jikun and Liu, Miao and Gupta, Abhinav and Pal, Christopher and Liu, Xue and Fu, Jie}, booktitle = {Proceedings of The 6th Conference on Robot Learning}, pages = {847--858}, year = {2023}, editor = {Liu, Karen and Kulic, Dana and Ichnowski, Jeff}, volume = {205}, series = {Proceedings of Machine Learning Research}, month = {14--18 Dec}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v205/kang23a/kang23a.pdf}, url = {https://proceedings.mlr.press/v205/kang23a.html}, abstract = {Various automatic curriculum learning (ACL) methods have been proposed to improve the sample efficiency and final performance of robots’ policies learning. They are designed to control how a robotic agent collects data, which is inspired by how humans gradually adapt their learning processes to their capabilities. In this paper, we propose a unified automatic curriculum learning framework to create multi-objective but coherent curricula that are generated by a set of parametric curriculum modules. Each curriculum module is instantiated as a neural network and is responsible for generating a particular curriculum. In order to coordinate those potentially conflicting modules in unified parameter space, we propose a multi-task hyper-net learning framework that uses a single hyper-net to parameterize all those curriculum modules. We evaluate our method on a series of robotic manipulation tasks and demonstrate its superiority over other state-of-the-art ACL methods in terms of sample efficiency and final performance.} }
Endnote
%0 Conference Paper %T Learning Multi-Objective Curricula for Robotic Policy Learning %A Jikun Kang %A Miao Liu %A Abhinav Gupta %A Christopher Pal %A Xue Liu %A Jie Fu %B Proceedings of The 6th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2023 %E Karen Liu %E Dana Kulic %E Jeff Ichnowski %F pmlr-v205-kang23a %I PMLR %P 847--858 %U https://proceedings.mlr.press/v205/kang23a.html %V 205 %X Various automatic curriculum learning (ACL) methods have been proposed to improve the sample efficiency and final performance of robots’ policies learning. They are designed to control how a robotic agent collects data, which is inspired by how humans gradually adapt their learning processes to their capabilities. In this paper, we propose a unified automatic curriculum learning framework to create multi-objective but coherent curricula that are generated by a set of parametric curriculum modules. Each curriculum module is instantiated as a neural network and is responsible for generating a particular curriculum. In order to coordinate those potentially conflicting modules in unified parameter space, we propose a multi-task hyper-net learning framework that uses a single hyper-net to parameterize all those curriculum modules. We evaluate our method on a series of robotic manipulation tasks and demonstrate its superiority over other state-of-the-art ACL methods in terms of sample efficiency and final performance.
APA
Kang, J., Liu, M., Gupta, A., Pal, C., Liu, X. & Fu, J.. (2023). Learning Multi-Objective Curricula for Robotic Policy Learning. Proceedings of The 6th Conference on Robot Learning, in Proceedings of Machine Learning Research 205:847-858 Available from https://proceedings.mlr.press/v205/kang23a.html.

Related Material