Learning RGB-D Feature Embeddings for Unseen Object Instance Segmentation

Yu Xiang, Christopher Xie, Arsalan Mousavian, Dieter Fox
Proceedings of the 2020 Conference on Robot Learning, PMLR 155:461-470, 2021.

Abstract

Segmenting unseen objects in cluttered scenes is an important skill that robots need to acquire in order to perform tasks in new environments. In this work, we propose a new method for unseen object instance segmentation by learning RGB-D feature embeddings from synthetic data. A metric learning loss function is utilized to learn to produce pixel-wise feature embeddings such that pixels from the same object are close to each other and pixels from different objects are separated in the embedding space. With the learned feature embeddings, a mean shift clustering algorithm can be applied to discover and segment unseen objects. We further improve the segmentation accuracy with a new two-stage clustering algorithm. Our method demonstrates that non-photorealistic synthetic RGB and depth images can be used to learn feature embeddings that transfer well to real-world images for unseen object instance segmentation.

Cite this Paper

BibTeX
@InProceedings{pmlr-v155-xiang21a, title = {Learning RGB-D Feature Embeddings for Unseen Object Instance Segmentation}, author = {Xiang, Yu and Xie, Christopher and Mousavian, Arsalan and Fox, Dieter}, booktitle = {Proceedings of the 2020 Conference on Robot Learning}, pages = {461--470}, year = {2021}, editor = {Kober, Jens and Ramos, Fabio and Tomlin, Claire}, volume = {155}, series = {Proceedings of Machine Learning Research}, month = {16--18 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v155/xiang21a/xiang21a.pdf}, url = {https://proceedings.mlr.press/v155/xiang21a.html}, abstract = {Segmenting unseen objects in cluttered scenes is an important skill that robots need to acquire in order to perform tasks in new environments. In this work, we propose a new method for unseen object instance segmentation by learning RGB-D feature embeddings from synthetic data. A metric learning loss function is utilized to learn to produce pixel-wise feature embeddings such that pixels from the same object are close to each other and pixels from different objects are separated in the embedding space. With the learned feature embeddings, a mean shift clustering algorithm can be applied to discover and segment unseen objects. We further improve the segmentation accuracy with a new two-stage clustering algorithm. Our method demonstrates that non-photorealistic synthetic RGB and depth images can be used to learn feature embeddings that transfer well to real-world images for unseen object instance segmentation.} }
Endnote
%0 Conference Paper %T Learning RGB-D Feature Embeddings for Unseen Object Instance Segmentation %A Yu Xiang %A Christopher Xie %A Arsalan Mousavian %A Dieter Fox %B Proceedings of the 2020 Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2021 %E Jens Kober %E Fabio Ramos %E Claire Tomlin %F pmlr-v155-xiang21a %I PMLR %P 461--470 %U https://proceedings.mlr.press/v155/xiang21a.html %V 155 %X Segmenting unseen objects in cluttered scenes is an important skill that robots need to acquire in order to perform tasks in new environments. In this work, we propose a new method for unseen object instance segmentation by learning RGB-D feature embeddings from synthetic data. A metric learning loss function is utilized to learn to produce pixel-wise feature embeddings such that pixels from the same object are close to each other and pixels from different objects are separated in the embedding space. With the learned feature embeddings, a mean shift clustering algorithm can be applied to discover and segment unseen objects. We further improve the segmentation accuracy with a new two-stage clustering algorithm. Our method demonstrates that non-photorealistic synthetic RGB and depth images can be used to learn feature embeddings that transfer well to real-world images for unseen object instance segmentation.
APA
Xiang, Y., Xie, C., Mousavian, A. & Fox, D.. (2021). Learning RGB-D Feature Embeddings for Unseen Object Instance Segmentation. Proceedings of the 2020 Conference on Robot Learning, in Proceedings of Machine Learning Research 155:461-470 Available from https://proceedings.mlr.press/v155/xiang21a.html.

Related Material