Amodal 3D Reconstruction for Robotic Manipulation via Stability and Connectivity

William Agnew, Christopher Xie, Aaron Walsman, Octavian Murad, Yubo Wang, Pedro Domingos), Siddhartha Srinivasa
Proceedings of the 2020 Conference on Robot Learning, PMLR 155:1498-1508, 2021.

Abstract

Learning-based 3D object reconstruction enables single- or few-shot estimation of 3D object models. For robotics, this holds the potential to allow model-based methods to rapidly adapt to novel objects and scenes. Existing 3D reconstruction techniques optimize for visual reconstruction fidelity, typically measured by chamfer distance or voxel IOU. We find that when applied to realistic, cluttered robotics environments, these systems produce reconstructions with low physical realism, resulting in poor task performance when used for model-based control. We propose ARM, an amodal 3D reconstruction system that introduces (1) a stability prior over object shapes, (2) a connectivity prior, and (3) a multi-channel input representation that allows for reasoning over relationships between groups of objects. By using these priors over the physical properties of objects, our system improves reconstruction quality not just by standard visual metrics, but also performance of model-based control on a variety of robotics manipulation tasks in challenging, cluttered environments.

Cite this Paper

BibTeX
@InProceedings{pmlr-v155-agnew21a, title = {Amodal 3D Reconstruction for Robotic Manipulation via Stability and Connectivity}, author = {Agnew, William and Xie, Christopher and Walsman, Aaron and Murad, Octavian and Wang, Yubo and Domingos), Pedro and Srinivasa, Siddhartha}, booktitle = {Proceedings of the 2020 Conference on Robot Learning}, pages = {1498--1508}, year = {2021}, editor = {Kober, Jens and Ramos, Fabio and Tomlin, Claire}, volume = {155}, series = {Proceedings of Machine Learning Research}, month = {16--18 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v155/agnew21a/agnew21a.pdf}, url = {https://proceedings.mlr.press/v155/agnew21a.html}, abstract = {Learning-based 3D object reconstruction enables single- or few-shot estimation of 3D object models. For robotics, this holds the potential to allow model-based methods to rapidly adapt to novel objects and scenes. Existing 3D reconstruction techniques optimize for visual reconstruction fidelity, typically measured by chamfer distance or voxel IOU. We find that when applied to realistic, cluttered robotics environments, these systems produce reconstructions with low physical realism, resulting in poor task performance when used for model-based control. We propose ARM, an amodal 3D reconstruction system that introduces (1) a stability prior over object shapes, (2) a connectivity prior, and (3) a multi-channel input representation that allows for reasoning over relationships between groups of objects. By using these priors over the physical properties of objects, our system improves reconstruction quality not just by standard visual metrics, but also performance of model-based control on a variety of robotics manipulation tasks in challenging, cluttered environments.} }
Endnote
%0 Conference Paper %T Amodal 3D Reconstruction for Robotic Manipulation via Stability and Connectivity %A William Agnew %A Christopher Xie %A Aaron Walsman %A Octavian Murad %A Yubo Wang %A Pedro Domingos) %A Siddhartha Srinivasa %B Proceedings of the 2020 Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2021 %E Jens Kober %E Fabio Ramos %E Claire Tomlin %F pmlr-v155-agnew21a %I PMLR %P 1498--1508 %U https://proceedings.mlr.press/v155/agnew21a.html %V 155 %X Learning-based 3D object reconstruction enables single- or few-shot estimation of 3D object models. For robotics, this holds the potential to allow model-based methods to rapidly adapt to novel objects and scenes. Existing 3D reconstruction techniques optimize for visual reconstruction fidelity, typically measured by chamfer distance or voxel IOU. We find that when applied to realistic, cluttered robotics environments, these systems produce reconstructions with low physical realism, resulting in poor task performance when used for model-based control. We propose ARM, an amodal 3D reconstruction system that introduces (1) a stability prior over object shapes, (2) a connectivity prior, and (3) a multi-channel input representation that allows for reasoning over relationships between groups of objects. By using these priors over the physical properties of objects, our system improves reconstruction quality not just by standard visual metrics, but also performance of model-based control on a variety of robotics manipulation tasks in challenging, cluttered environments.
APA
Agnew, W., Xie, C., Walsman, A., Murad, O., Wang, Y., Domingos), P. & Srinivasa, S.. (2021). Amodal 3D Reconstruction for Robotic Manipulation via Stability and Connectivity. Proceedings of the 2020 Conference on Robot Learning, in Proceedings of Machine Learning Research 155:1498-1508 Available from https://proceedings.mlr.press/v155/agnew21a.html.

Related Material