Learning Reusable Manipulation Strategies

Jiayuan Mao, Tomás Lozano-Pérez, Joshua B. Tenenbaum, Leslie Pack Kaelbling
Proceedings of The 7th Conference on Robot Learning, PMLR 229:1467-1483, 2023.

Abstract

Humans demonstrate an impressive ability to acquire and generalize manipulation “tricks.” Even from a single demonstration, such as using soup ladles to reach for distant objects, we can apply this skill to new scenarios involving different object positions, sizes, and categories (e.g., forks and hammers). Additionally, we can flexibly combine various skills to devise long-term plans. In this paper, we present a framework that enables machines to acquire such manipulation skills, referred to as “mechanisms,” through a single demonstration and self-play. Our key insight lies in interpreting each demonstration as a sequence of changes in robot-object and object-object contact modes, which provides a scaffold for learning detailed samplers for continuous parameters. These learned mechanisms and samplers can be seamlessly integrated into standard task and motion planners, enabling their compositional use.

Cite this Paper

BibTeX
@InProceedings{pmlr-v229-mao23a, title = {Learning Reusable Manipulation Strategies}, author = {Mao, Jiayuan and Lozano-P\'{e}rez, Tom\'{a}s and Tenenbaum, Joshua B. and Kaelbling, Leslie Pack}, booktitle = {Proceedings of The 7th Conference on Robot Learning}, pages = {1467--1483}, year = {2023}, editor = {Tan, Jie and Toussaint, Marc and Darvish, Kourosh}, volume = {229}, series = {Proceedings of Machine Learning Research}, month = {06--09 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v229/mao23a/mao23a.pdf}, url = {https://proceedings.mlr.press/v229/mao23a.html}, abstract = {Humans demonstrate an impressive ability to acquire and generalize manipulation “tricks.” Even from a single demonstration, such as using soup ladles to reach for distant objects, we can apply this skill to new scenarios involving different object positions, sizes, and categories (e.g., forks and hammers). Additionally, we can flexibly combine various skills to devise long-term plans. In this paper, we present a framework that enables machines to acquire such manipulation skills, referred to as “mechanisms,” through a single demonstration and self-play. Our key insight lies in interpreting each demonstration as a sequence of changes in robot-object and object-object contact modes, which provides a scaffold for learning detailed samplers for continuous parameters. These learned mechanisms and samplers can be seamlessly integrated into standard task and motion planners, enabling their compositional use.} }
Endnote
%0 Conference Paper %T Learning Reusable Manipulation Strategies %A Jiayuan Mao %A Tomás Lozano-Pérez %A Joshua B. Tenenbaum %A Leslie Pack Kaelbling %B Proceedings of The 7th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2023 %E Jie Tan %E Marc Toussaint %E Kourosh Darvish %F pmlr-v229-mao23a %I PMLR %P 1467--1483 %U https://proceedings.mlr.press/v229/mao23a.html %V 229 %X Humans demonstrate an impressive ability to acquire and generalize manipulation “tricks.” Even from a single demonstration, such as using soup ladles to reach for distant objects, we can apply this skill to new scenarios involving different object positions, sizes, and categories (e.g., forks and hammers). Additionally, we can flexibly combine various skills to devise long-term plans. In this paper, we present a framework that enables machines to acquire such manipulation skills, referred to as “mechanisms,” through a single demonstration and self-play. Our key insight lies in interpreting each demonstration as a sequence of changes in robot-object and object-object contact modes, which provides a scaffold for learning detailed samplers for continuous parameters. These learned mechanisms and samplers can be seamlessly integrated into standard task and motion planners, enabling their compositional use.
APA
Mao, J., Lozano-Pérez, T., Tenenbaum, J.B. & Kaelbling, L.P.. (2023). Learning Reusable Manipulation Strategies. Proceedings of The 7th Conference on Robot Learning, in Proceedings of Machine Learning Research 229:1467-1483 Available from https://proceedings.mlr.press/v229/mao23a.html.

Related Material