CLASP: Constrained Latent Shape Projection for Refining Object Shape from Robot Contact

Brad Saund, Dmitry Berenson
Proceedings of the 5th Conference on Robot Learning, PMLR 164:1391-1400, 2022.

Abstract

Robots need both visual and contact sensing to effectively estimate the state of their environment. Camera RGBD data provides rich information of the objects surrounding the robot, and shape priors can help correct noise and fill in gaps and occluded regions. However, when the robot senses unexpected contact, the estimate should be updated to explain the contact. To address this need, we propose CLASP: Constrained Latent Shape Projection. This approach consists of a shape completion network that generates a prior from RGBD data and a procedure to generate shapes consistent with both the network prior and robot contact observations. We find CLASP consistently decreases the Chamfer Distance between the predicted and ground truth scenes, while other approaches do not benefit from contact information.

Cite this Paper

BibTeX
@InProceedings{pmlr-v164-saund22a, title = {CLASP: Constrained Latent Shape Projection for Refining Object Shape from Robot Contact}, author = {Saund, Brad and Berenson, Dmitry}, booktitle = {Proceedings of the 5th Conference on Robot Learning}, pages = {1391--1400}, year = {2022}, editor = {Faust, Aleksandra and Hsu, David and Neumann, Gerhard}, volume = {164}, series = {Proceedings of Machine Learning Research}, month = {08--11 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v164/saund22a/saund22a.pdf}, url = {https://proceedings.mlr.press/v164/saund22a.html}, abstract = {Robots need both visual and contact sensing to effectively estimate the state of their environment. Camera RGBD data provides rich information of the objects surrounding the robot, and shape priors can help correct noise and fill in gaps and occluded regions. However, when the robot senses unexpected contact, the estimate should be updated to explain the contact. To address this need, we propose CLASP: Constrained Latent Shape Projection. This approach consists of a shape completion network that generates a prior from RGBD data and a procedure to generate shapes consistent with both the network prior and robot contact observations. We find CLASP consistently decreases the Chamfer Distance between the predicted and ground truth scenes, while other approaches do not benefit from contact information.} }
Endnote
%0 Conference Paper %T CLASP: Constrained Latent Shape Projection for Refining Object Shape from Robot Contact %A Brad Saund %A Dmitry Berenson %B Proceedings of the 5th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2022 %E Aleksandra Faust %E David Hsu %E Gerhard Neumann %F pmlr-v164-saund22a %I PMLR %P 1391--1400 %U https://proceedings.mlr.press/v164/saund22a.html %V 164 %X Robots need both visual and contact sensing to effectively estimate the state of their environment. Camera RGBD data provides rich information of the objects surrounding the robot, and shape priors can help correct noise and fill in gaps and occluded regions. However, when the robot senses unexpected contact, the estimate should be updated to explain the contact. To address this need, we propose CLASP: Constrained Latent Shape Projection. This approach consists of a shape completion network that generates a prior from RGBD data and a procedure to generate shapes consistent with both the network prior and robot contact observations. We find CLASP consistently decreases the Chamfer Distance between the predicted and ground truth scenes, while other approaches do not benefit from contact information.
APA
Saund, B. & Berenson, D.. (2022). CLASP: Constrained Latent Shape Projection for Refining Object Shape from Robot Contact. Proceedings of the 5th Conference on Robot Learning, in Proceedings of Machine Learning Research 164:1391-1400 Available from https://proceedings.mlr.press/v164/saund22a.html.

Related Material