One-shot Imitation Learning via Interaction Warping

Ondrej Biza, Skye Thompson, Kishore Reddy Pagidi, Abhinav Kumar, Elise van der Pol, Robin Walters, Thomas Kipf, Jan-Willem van de Meent, Lawson L. S. Wong, Robert Platt
Proceedings of The 7th Conference on Robot Learning, PMLR 229:2519-2536, 2023.

Abstract

Learning robot policies from few demonstrations is crucial in open-ended applications. We propose a new method, Interaction Warping, for one-shot learning SE(3) robotic manipulation policies. We infer the 3D mesh of each object in the environment using shape warping, a technique for aligning point clouds across object instances. Then, we represent manipulation actions as keypoints on objects, which can be warped with the shape of the object. We show successful one-shot imitation learning on three simulated and real-world object re-arrangement tasks. We also demonstrate the ability of our method to predict object meshes and robot grasps in the wild. Webpage: https://shapewarping.github.io.

Cite this Paper

BibTeX
@InProceedings{pmlr-v229-biza23a, title = {One-shot Imitation Learning via Interaction Warping}, author = {Biza, Ondrej and Thompson, Skye and Pagidi, Kishore Reddy and Kumar, Abhinav and Pol, Elise van der and Walters, Robin and Kipf, Thomas and Meent, Jan-Willem van de and Wong, Lawson L. S. and Platt, Robert}, booktitle = {Proceedings of The 7th Conference on Robot Learning}, pages = {2519--2536}, year = {2023}, editor = {Tan, Jie and Toussaint, Marc and Darvish, Kourosh}, volume = {229}, series = {Proceedings of Machine Learning Research}, month = {06--09 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v229/biza23a/biza23a.pdf}, url = {https://proceedings.mlr.press/v229/biza23a.html}, abstract = {Learning robot policies from few demonstrations is crucial in open-ended applications. We propose a new method, Interaction Warping, for one-shot learning SE(3) robotic manipulation policies. We infer the 3D mesh of each object in the environment using shape warping, a technique for aligning point clouds across object instances. Then, we represent manipulation actions as keypoints on objects, which can be warped with the shape of the object. We show successful one-shot imitation learning on three simulated and real-world object re-arrangement tasks. We also demonstrate the ability of our method to predict object meshes and robot grasps in the wild. Webpage: https://shapewarping.github.io.} }
Endnote
%0 Conference Paper %T One-shot Imitation Learning via Interaction Warping %A Ondrej Biza %A Skye Thompson %A Kishore Reddy Pagidi %A Abhinav Kumar %A Elise van der Pol %A Robin Walters %A Thomas Kipf %A Jan-Willem van de Meent %A Lawson L. S. Wong %A Robert Platt %B Proceedings of The 7th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2023 %E Jie Tan %E Marc Toussaint %E Kourosh Darvish %F pmlr-v229-biza23a %I PMLR %P 2519--2536 %U https://proceedings.mlr.press/v229/biza23a.html %V 229 %X Learning robot policies from few demonstrations is crucial in open-ended applications. We propose a new method, Interaction Warping, for one-shot learning SE(3) robotic manipulation policies. We infer the 3D mesh of each object in the environment using shape warping, a technique for aligning point clouds across object instances. Then, we represent manipulation actions as keypoints on objects, which can be warped with the shape of the object. We show successful one-shot imitation learning on three simulated and real-world object re-arrangement tasks. We also demonstrate the ability of our method to predict object meshes and robot grasps in the wild. Webpage: https://shapewarping.github.io.
APA
Biza, O., Thompson, S., Pagidi, K.R., Kumar, A., Pol, E.v.d., Walters, R., Kipf, T., Meent, J.v.d., Wong, L.L.S. & Platt, R.. (2023). One-shot Imitation Learning via Interaction Warping. Proceedings of The 7th Conference on Robot Learning, in Proceedings of Machine Learning Research 229:2519-2536 Available from https://proceedings.mlr.press/v229/biza23a.html.

Related Material