Learning Robot Manipulation from Cross-Morphology Demonstration

Gautam Salhotra, I-Chun Arthur Liu, Gaurav S. Sukhatme
Proceedings of The 7th Conference on Robot Learning, PMLR 229:2257-2277, 2023.

Abstract

Some Learning from Demonstrations (LfD) methods handle small mismatches in the action spaces of the teacher and student. Here we address the casewhere the teacher’s morphology is substantially different from that of the student. Our framework, Morphological Adaptation in Imitation Learning (MAIL), bridges this gap allowing us to train an agent from demonstrations by other agents with significantly different morphologies. MAIL learns from suboptimal demonstrations, so long as they provide some guidance towards a desired solution. We demonstrate MAIL on manipulation tasks with rigid and deformable objects including 3D cloth manipulation interacting with rigid obstacles. We train a visual control policy for a robot with one end-effector using demonstrations from a simulated agent with two end-effectors. MAIL shows up to $24%$ improvement in a normalized performance metric over LfD and non-LfD baselines. It is deployed to a real Franka Panda robot, handles multiple variations in properties for objects (size, rotation, translation), and cloth-specific properties (color, thickness, size, material).

Cite this Paper

BibTeX
@InProceedings{pmlr-v229-salhotra23a, title = {Learning Robot Manipulation from Cross-Morphology Demonstration}, author = {Salhotra, Gautam and Liu, I-Chun Arthur and Sukhatme, Gaurav S.}, booktitle = {Proceedings of The 7th Conference on Robot Learning}, pages = {2257--2277}, year = {2023}, editor = {Tan, Jie and Toussaint, Marc and Darvish, Kourosh}, volume = {229}, series = {Proceedings of Machine Learning Research}, month = {06--09 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v229/salhotra23a/salhotra23a.pdf}, url = {https://proceedings.mlr.press/v229/salhotra23a.html}, abstract = {Some Learning from Demonstrations (LfD) methods handle small mismatches in the action spaces of the teacher and student. Here we address the casewhere the teacher’s morphology is substantially different from that of the student. Our framework, Morphological Adaptation in Imitation Learning (MAIL), bridges this gap allowing us to train an agent from demonstrations by other agents with significantly different morphologies. MAIL learns from suboptimal demonstrations, so long as they provide some guidance towards a desired solution. We demonstrate MAIL on manipulation tasks with rigid and deformable objects including 3D cloth manipulation interacting with rigid obstacles. We train a visual control policy for a robot with one end-effector using demonstrations from a simulated agent with two end-effectors. MAIL shows up to $24%$ improvement in a normalized performance metric over LfD and non-LfD baselines. It is deployed to a real Franka Panda robot, handles multiple variations in properties for objects (size, rotation, translation), and cloth-specific properties (color, thickness, size, material).} }
Endnote
%0 Conference Paper %T Learning Robot Manipulation from Cross-Morphology Demonstration %A Gautam Salhotra %A I-Chun Arthur Liu %A Gaurav S. Sukhatme %B Proceedings of The 7th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2023 %E Jie Tan %E Marc Toussaint %E Kourosh Darvish %F pmlr-v229-salhotra23a %I PMLR %P 2257--2277 %U https://proceedings.mlr.press/v229/salhotra23a.html %V 229 %X Some Learning from Demonstrations (LfD) methods handle small mismatches in the action spaces of the teacher and student. Here we address the casewhere the teacher’s morphology is substantially different from that of the student. Our framework, Morphological Adaptation in Imitation Learning (MAIL), bridges this gap allowing us to train an agent from demonstrations by other agents with significantly different morphologies. MAIL learns from suboptimal demonstrations, so long as they provide some guidance towards a desired solution. We demonstrate MAIL on manipulation tasks with rigid and deformable objects including 3D cloth manipulation interacting with rigid obstacles. We train a visual control policy for a robot with one end-effector using demonstrations from a simulated agent with two end-effectors. MAIL shows up to $24%$ improvement in a normalized performance metric over LfD and non-LfD baselines. It is deployed to a real Franka Panda robot, handles multiple variations in properties for objects (size, rotation, translation), and cloth-specific properties (color, thickness, size, material).
APA
Salhotra, G., Liu, I.A. & Sukhatme, G.S.. (2023). Learning Robot Manipulation from Cross-Morphology Demonstration. Proceedings of The 7th Conference on Robot Learning, in Proceedings of Machine Learning Research 229:2257-2277 Available from https://proceedings.mlr.press/v229/salhotra23a.html.

Related Material