Learning Models as Functionals of Signed-Distance Fields for Manipulation Planning

Danny Driess, Jung-Su Ha, Marc Toussaint, Russ Tedrake
Proceedings of the 5th Conference on Robot Learning, PMLR 164:245-255, 2022.

Abstract

This work proposes an optimization-based manipulation planning framework where the objectives are learned functionals of signed-distance fields that represent objects in the scene. Most manipulation planning approaches rely on analytical models and carefully chosen abstractions/state-spaces to be effective. A central question is how models can be obtained from data that are not primarily accurate in their predictions, but, more importantly, enable efficient reasoning within a planning framework, while at the same time being closely coupled to perception spaces. We show that representing objects as signed-distance fields not only enables to learn and represent a variety of models with higher accuracy compared to point-cloud and occupancy measure representations, but also that SDF-based models are suitable for optimization-based planning. To demonstrate the versatility of our approach, we learn both kinematic and dynamic models to solve tasks that involve hanging mugs on hooks and pushing objects on a table. We can unify these quite different tasks within one framework, since SDFs are the common object representation. Video: https://youtu.be/ga8Wlkss7co

Cite this Paper

BibTeX
@InProceedings{pmlr-v164-driess22a, title = {Learning Models as Functionals of Signed-Distance Fields for Manipulation Planning}, author = {Driess, Danny and Ha, Jung-Su and Toussaint, Marc and Tedrake, Russ}, booktitle = {Proceedings of the 5th Conference on Robot Learning}, pages = {245--255}, year = {2022}, editor = {Faust, Aleksandra and Hsu, David and Neumann, Gerhard}, volume = {164}, series = {Proceedings of Machine Learning Research}, month = {08--11 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v164/driess22a/driess22a.pdf}, url = {https://proceedings.mlr.press/v164/driess22a.html}, abstract = {This work proposes an optimization-based manipulation planning framework where the objectives are learned functionals of signed-distance fields that represent objects in the scene. Most manipulation planning approaches rely on analytical models and carefully chosen abstractions/state-spaces to be effective. A central question is how models can be obtained from data that are not primarily accurate in their predictions, but, more importantly, enable efficient reasoning within a planning framework, while at the same time being closely coupled to perception spaces. We show that representing objects as signed-distance fields not only enables to learn and represent a variety of models with higher accuracy compared to point-cloud and occupancy measure representations, but also that SDF-based models are suitable for optimization-based planning. To demonstrate the versatility of our approach, we learn both kinematic and dynamic models to solve tasks that involve hanging mugs on hooks and pushing objects on a table. We can unify these quite different tasks within one framework, since SDFs are the common object representation. Video: https://youtu.be/ga8Wlkss7co} }
Endnote
%0 Conference Paper %T Learning Models as Functionals of Signed-Distance Fields for Manipulation Planning %A Danny Driess %A Jung-Su Ha %A Marc Toussaint %A Russ Tedrake %B Proceedings of the 5th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2022 %E Aleksandra Faust %E David Hsu %E Gerhard Neumann %F pmlr-v164-driess22a %I PMLR %P 245--255 %U https://proceedings.mlr.press/v164/driess22a.html %V 164 %X This work proposes an optimization-based manipulation planning framework where the objectives are learned functionals of signed-distance fields that represent objects in the scene. Most manipulation planning approaches rely on analytical models and carefully chosen abstractions/state-spaces to be effective. A central question is how models can be obtained from data that are not primarily accurate in their predictions, but, more importantly, enable efficient reasoning within a planning framework, while at the same time being closely coupled to perception spaces. We show that representing objects as signed-distance fields not only enables to learn and represent a variety of models with higher accuracy compared to point-cloud and occupancy measure representations, but also that SDF-based models are suitable for optimization-based planning. To demonstrate the versatility of our approach, we learn both kinematic and dynamic models to solve tasks that involve hanging mugs on hooks and pushing objects on a table. We can unify these quite different tasks within one framework, since SDFs are the common object representation. Video: https://youtu.be/ga8Wlkss7co
APA
Driess, D., Ha, J., Toussaint, M. & Tedrake, R.. (2022). Learning Models as Functionals of Signed-Distance Fields for Manipulation Planning. Proceedings of the 5th Conference on Robot Learning, in Proceedings of Machine Learning Research 164:245-255 Available from https://proceedings.mlr.press/v164/driess22a.html.

Related Material