STOW: Discrete-Frame Segmentation and Tracking of Unseen Objects for Warehouse Picking Robots

Yi Li, Muru Zhang, Markus Grotz, Kaichun Mo, Dieter Fox
Proceedings of The 7th Conference on Robot Learning, PMLR 229:3732-3748, 2023.

Abstract

Segmentation and tracking of unseen object instances in discrete frames pose a significant challenge in dynamic industrial robotic contexts, such as distribution warehouses. Here, robots must handle object rearrangements, including shifting, removal, and partial occlusion by new items, and track these items after substantial temporal gaps. The task is further complicated when robots encounter objects beyond their training sets, thereby requiring the ability to segment and track previously unseen items. Considering that continuous observation is often inaccessible in such settings, our task involves working with a discrete set of frames separated by indefinite periods, during which substantial changes to the scene may occur. This task also translates to domestic robotic applications, such as table rearrangement. To address these demanding challenges, we introduce new synthetic and real-world datasets that replicate these industrial and household scenarios. Furthermore, we propose a novel paradigm for joint segmentation and tracking in discrete frames, alongside a transformer module that facilitates efficient inter-frame communication. Our approach significantly outperforms recent methods in our experiments. For additional results and videos, please visit https://sites.google.com/view/stow-corl23. Code and dataset will be released.

Cite this Paper

BibTeX
@InProceedings{pmlr-v229-li23c, title = {STOW: Discrete-Frame Segmentation and Tracking of Unseen Objects for Warehouse Picking Robots}, author = {Li, Yi and Zhang, Muru and Grotz, Markus and Mo, Kaichun and Fox, Dieter}, booktitle = {Proceedings of The 7th Conference on Robot Learning}, pages = {3732--3748}, year = {2023}, editor = {Tan, Jie and Toussaint, Marc and Darvish, Kourosh}, volume = {229}, series = {Proceedings of Machine Learning Research}, month = {06--09 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v229/li23c/li23c.pdf}, url = {https://proceedings.mlr.press/v229/li23c.html}, abstract = {Segmentation and tracking of unseen object instances in discrete frames pose a significant challenge in dynamic industrial robotic contexts, such as distribution warehouses. Here, robots must handle object rearrangements, including shifting, removal, and partial occlusion by new items, and track these items after substantial temporal gaps. The task is further complicated when robots encounter objects beyond their training sets, thereby requiring the ability to segment and track previously unseen items. Considering that continuous observation is often inaccessible in such settings, our task involves working with a discrete set of frames separated by indefinite periods, during which substantial changes to the scene may occur. This task also translates to domestic robotic applications, such as table rearrangement. To address these demanding challenges, we introduce new synthetic and real-world datasets that replicate these industrial and household scenarios. Furthermore, we propose a novel paradigm for joint segmentation and tracking in discrete frames, alongside a transformer module that facilitates efficient inter-frame communication. Our approach significantly outperforms recent methods in our experiments. For additional results and videos, please visit https://sites.google.com/view/stow-corl23. Code and dataset will be released.} }
Endnote
%0 Conference Paper %T STOW: Discrete-Frame Segmentation and Tracking of Unseen Objects for Warehouse Picking Robots %A Yi Li %A Muru Zhang %A Markus Grotz %A Kaichun Mo %A Dieter Fox %B Proceedings of The 7th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2023 %E Jie Tan %E Marc Toussaint %E Kourosh Darvish %F pmlr-v229-li23c %I PMLR %P 3732--3748 %U https://proceedings.mlr.press/v229/li23c.html %V 229 %X Segmentation and tracking of unseen object instances in discrete frames pose a significant challenge in dynamic industrial robotic contexts, such as distribution warehouses. Here, robots must handle object rearrangements, including shifting, removal, and partial occlusion by new items, and track these items after substantial temporal gaps. The task is further complicated when robots encounter objects beyond their training sets, thereby requiring the ability to segment and track previously unseen items. Considering that continuous observation is often inaccessible in such settings, our task involves working with a discrete set of frames separated by indefinite periods, during which substantial changes to the scene may occur. This task also translates to domestic robotic applications, such as table rearrangement. To address these demanding challenges, we introduce new synthetic and real-world datasets that replicate these industrial and household scenarios. Furthermore, we propose a novel paradigm for joint segmentation and tracking in discrete frames, alongside a transformer module that facilitates efficient inter-frame communication. Our approach significantly outperforms recent methods in our experiments. For additional results and videos, please visit https://sites.google.com/view/stow-corl23. Code and dataset will be released.
APA
Li, Y., Zhang, M., Grotz, M., Mo, K. & Fox, D.. (2023). STOW: Discrete-Frame Segmentation and Tracking of Unseen Objects for Warehouse Picking Robots. Proceedings of The 7th Conference on Robot Learning, in Proceedings of Machine Learning Research 229:3732-3748 Available from https://proceedings.mlr.press/v229/li23c.html.

Related Material