HERD: Continuous Human-to-Robot Evolution for Learning from Human Demonstration

Xingyu Liu, Deepak Pathak, Kris M. Kitani
Proceedings of The 6th Conference on Robot Learning, PMLR 205:447-458, 2023.

Abstract

The ability to learn from human demonstration endows robots with the ability to automate various tasks. However, directly learning from human demonstration is challenging since the structure of the human hand can be very different from the desired robot gripper. In this work, we show that manipulation skills can be transferred from a human to a robot through the use of micro-evolutionary reinforcement learning, where a five-finger human dexterous hand robot gradually evolves into a commercial robot, while repeated interacting in a physics simulator to continuously update the policy that is first learned from human demonstration. To deal with the high dimensions of robot parameters, we propose an algorithm for multi-dimensional evolution path searching that allows joint optimization of both the robot evolution path and the policy. Through experiments on human object manipulation datasets, we show that our framework can efficiently transfer the expert human agent policy trained from human demonstrations in diverse modalities to target commercial robots.

Cite this Paper

BibTeX
@InProceedings{pmlr-v205-liu23b, title = {HERD: Continuous Human-to-Robot Evolution for Learning from Human Demonstration}, author = {Liu, Xingyu and Pathak, Deepak and Kitani, Kris M.}, booktitle = {Proceedings of The 6th Conference on Robot Learning}, pages = {447--458}, year = {2023}, editor = {Liu, Karen and Kulic, Dana and Ichnowski, Jeff}, volume = {205}, series = {Proceedings of Machine Learning Research}, month = {14--18 Dec}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v205/liu23b/liu23b.pdf}, url = {https://proceedings.mlr.press/v205/liu23b.html}, abstract = {The ability to learn from human demonstration endows robots with the ability to automate various tasks. However, directly learning from human demonstration is challenging since the structure of the human hand can be very different from the desired robot gripper. In this work, we show that manipulation skills can be transferred from a human to a robot through the use of micro-evolutionary reinforcement learning, where a five-finger human dexterous hand robot gradually evolves into a commercial robot, while repeated interacting in a physics simulator to continuously update the policy that is first learned from human demonstration. To deal with the high dimensions of robot parameters, we propose an algorithm for multi-dimensional evolution path searching that allows joint optimization of both the robot evolution path and the policy. Through experiments on human object manipulation datasets, we show that our framework can efficiently transfer the expert human agent policy trained from human demonstrations in diverse modalities to target commercial robots.} }
Endnote
%0 Conference Paper %T HERD: Continuous Human-to-Robot Evolution for Learning from Human Demonstration %A Xingyu Liu %A Deepak Pathak %A Kris M. Kitani %B Proceedings of The 6th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2023 %E Karen Liu %E Dana Kulic %E Jeff Ichnowski %F pmlr-v205-liu23b %I PMLR %P 447--458 %U https://proceedings.mlr.press/v205/liu23b.html %V 205 %X The ability to learn from human demonstration endows robots with the ability to automate various tasks. However, directly learning from human demonstration is challenging since the structure of the human hand can be very different from the desired robot gripper. In this work, we show that manipulation skills can be transferred from a human to a robot through the use of micro-evolutionary reinforcement learning, where a five-finger human dexterous hand robot gradually evolves into a commercial robot, while repeated interacting in a physics simulator to continuously update the policy that is first learned from human demonstration. To deal with the high dimensions of robot parameters, we propose an algorithm for multi-dimensional evolution path searching that allows joint optimization of both the robot evolution path and the policy. Through experiments on human object manipulation datasets, we show that our framework can efficiently transfer the expert human agent policy trained from human demonstrations in diverse modalities to target commercial robots.
APA
Liu, X., Pathak, D. & Kitani, K.M.. (2023). HERD: Continuous Human-to-Robot Evolution for Learning from Human Demonstration. Proceedings of The 6th Conference on Robot Learning, in Proceedings of Machine Learning Research 205:447-458 Available from https://proceedings.mlr.press/v205/liu23b.html.

Related Material