Predicting Stable Configurations for Semantic Placement of Novel Objects

Chris Paxton, Chris Xie, Tucker Hermans, Dieter Fox
Proceedings of the 5th Conference on Robot Learning, PMLR 164:806-815, 2022.

Abstract

Human environments contain numerous objects configured in a variety of arrangements. Our goal is to enable robots to repose previously unseen objects according to learned semantic relationships in novel environments. We break this problem down into two parts: (1) finding physically valid locations for the objects and (2) determining if those poses satisfy learned, high-level semantic relationships. We build our models and training from the ground up to be tightly integrated with our proposed planning algorithm for semantic placement of unknown objects. We train our models purely in simulation, with no fine-tuning needed for use in the real world. Our approach enables motion planning for semantic rearrangement of unknown objects in scenes with varying geometry from only RGB-D sensing. Our experiments through a set of simulated ablations demonstrate that using a relational classifier alone is not sufficient for reliable planning. We further demonstrate the ability of our planner to generate and execute diverse manipulation plans through a set of real-world experiments with a variety of objects.

Cite this Paper

BibTeX
@InProceedings{pmlr-v164-paxton22a, title = {Predicting Stable Configurations for Semantic Placement of Novel Objects}, author = {Paxton, Chris and Xie, Chris and Hermans, Tucker and Fox, Dieter}, booktitle = {Proceedings of the 5th Conference on Robot Learning}, pages = {806--815}, year = {2022}, editor = {Faust, Aleksandra and Hsu, David and Neumann, Gerhard}, volume = {164}, series = {Proceedings of Machine Learning Research}, month = {08--11 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v164/paxton22a/paxton22a.pdf}, url = {https://proceedings.mlr.press/v164/paxton22a.html}, abstract = {Human environments contain numerous objects configured in a variety of arrangements. Our goal is to enable robots to repose previously unseen objects according to learned semantic relationships in novel environments. We break this problem down into two parts: (1) finding physically valid locations for the objects and (2) determining if those poses satisfy learned, high-level semantic relationships. We build our models and training from the ground up to be tightly integrated with our proposed planning algorithm for semantic placement of unknown objects. We train our models purely in simulation, with no fine-tuning needed for use in the real world. Our approach enables motion planning for semantic rearrangement of unknown objects in scenes with varying geometry from only RGB-D sensing. Our experiments through a set of simulated ablations demonstrate that using a relational classifier alone is not sufficient for reliable planning. We further demonstrate the ability of our planner to generate and execute diverse manipulation plans through a set of real-world experiments with a variety of objects.} }
Endnote
%0 Conference Paper %T Predicting Stable Configurations for Semantic Placement of Novel Objects %A Chris Paxton %A Chris Xie %A Tucker Hermans %A Dieter Fox %B Proceedings of the 5th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2022 %E Aleksandra Faust %E David Hsu %E Gerhard Neumann %F pmlr-v164-paxton22a %I PMLR %P 806--815 %U https://proceedings.mlr.press/v164/paxton22a.html %V 164 %X Human environments contain numerous objects configured in a variety of arrangements. Our goal is to enable robots to repose previously unseen objects according to learned semantic relationships in novel environments. We break this problem down into two parts: (1) finding physically valid locations for the objects and (2) determining if those poses satisfy learned, high-level semantic relationships. We build our models and training from the ground up to be tightly integrated with our proposed planning algorithm for semantic placement of unknown objects. We train our models purely in simulation, with no fine-tuning needed for use in the real world. Our approach enables motion planning for semantic rearrangement of unknown objects in scenes with varying geometry from only RGB-D sensing. Our experiments through a set of simulated ablations demonstrate that using a relational classifier alone is not sufficient for reliable planning. We further demonstrate the ability of our planner to generate and execute diverse manipulation plans through a set of real-world experiments with a variety of objects.
APA
Paxton, C., Xie, C., Hermans, T. & Fox, D.. (2022). Predicting Stable Configurations for Semantic Placement of Novel Objects. Proceedings of the 5th Conference on Robot Learning, in Proceedings of Machine Learning Research 164:806-815 Available from https://proceedings.mlr.press/v164/paxton22a.html.

Related Material