RICE: Refining Instance Masks in Cluttered Environments with Graph Neural Networks

Chris Xie, Arsalan Mousavian, Yu Xiang, Dieter Fox
Proceedings of the 5th Conference on Robot Learning, PMLR 164:1655-1665, 2022.

Abstract

Segmenting unseen object instances in cluttered environments is an important capability that robots need when functioning in unstructured environments. While previous methods have exhibited promising results, they still tend to provide incorrect results in highly cluttered scenes. We postulate that a network architecture that encodes relations between objects at a high-level can be beneficial. Thus, in this work, we propose a novel framework that refines the output of such methods by utilizing a graph-based representation of instance masks. We train deep networks capable of sampling smart perturbations to the segmentations, and a graph neural network, which can encode relations between objects, to evaluate the perturbed segmentations. Our proposed method is orthogonal to previous works and achieves state-of-the-art performance when combined with them. We demonstrate an application that uses uncertainty estimates generated by our method to guide a manipulator, leading to efficient understanding of cluttered scenes. Code, models, and video can be found at https://github.com/chrisdxie/rice.

Cite this Paper

BibTeX
@InProceedings{pmlr-v164-xie22a, title = {RICE: Refining Instance Masks in Cluttered Environments with Graph Neural Networks}, author = {Xie, Chris and Mousavian, Arsalan and Xiang, Yu and Fox, Dieter}, booktitle = {Proceedings of the 5th Conference on Robot Learning}, pages = {1655--1665}, year = {2022}, editor = {Faust, Aleksandra and Hsu, David and Neumann, Gerhard}, volume = {164}, series = {Proceedings of Machine Learning Research}, month = {08--11 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v164/xie22a/xie22a.pdf}, url = {https://proceedings.mlr.press/v164/xie22a.html}, abstract = {Segmenting unseen object instances in cluttered environments is an important capability that robots need when functioning in unstructured environments. While previous methods have exhibited promising results, they still tend to provide incorrect results in highly cluttered scenes. We postulate that a network architecture that encodes relations between objects at a high-level can be beneficial. Thus, in this work, we propose a novel framework that refines the output of such methods by utilizing a graph-based representation of instance masks. We train deep networks capable of sampling smart perturbations to the segmentations, and a graph neural network, which can encode relations between objects, to evaluate the perturbed segmentations. Our proposed method is orthogonal to previous works and achieves state-of-the-art performance when combined with them. We demonstrate an application that uses uncertainty estimates generated by our method to guide a manipulator, leading to efficient understanding of cluttered scenes. Code, models, and video can be found at https://github.com/chrisdxie/rice.} }
Endnote
%0 Conference Paper %T RICE: Refining Instance Masks in Cluttered Environments with Graph Neural Networks %A Chris Xie %A Arsalan Mousavian %A Yu Xiang %A Dieter Fox %B Proceedings of the 5th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2022 %E Aleksandra Faust %E David Hsu %E Gerhard Neumann %F pmlr-v164-xie22a %I PMLR %P 1655--1665 %U https://proceedings.mlr.press/v164/xie22a.html %V 164 %X Segmenting unseen object instances in cluttered environments is an important capability that robots need when functioning in unstructured environments. While previous methods have exhibited promising results, they still tend to provide incorrect results in highly cluttered scenes. We postulate that a network architecture that encodes relations between objects at a high-level can be beneficial. Thus, in this work, we propose a novel framework that refines the output of such methods by utilizing a graph-based representation of instance masks. We train deep networks capable of sampling smart perturbations to the segmentations, and a graph neural network, which can encode relations between objects, to evaluate the perturbed segmentations. Our proposed method is orthogonal to previous works and achieves state-of-the-art performance when combined with them. We demonstrate an application that uses uncertainty estimates generated by our method to guide a manipulator, leading to efficient understanding of cluttered scenes. Code, models, and video can be found at https://github.com/chrisdxie/rice.
APA
Xie, C., Mousavian, A., Xiang, Y. & Fox, D.. (2022). RICE: Refining Instance Masks in Cluttered Environments with Graph Neural Networks. Proceedings of the 5th Conference on Robot Learning, in Proceedings of Machine Learning Research 164:1655-1665 Available from https://proceedings.mlr.press/v164/xie22a.html.

Related Material