Iterative Interactive Modeling for Knotting Plastic Bags

Chongkai Gao, Zekun Li, Haichuan Gao, Feng Chen
Proceedings of The 6th Conference on Robot Learning, PMLR 205:571-582, 2023.

Abstract

Deformable object manipulation has great research significance for the robotic community and numerous applications in daily life. In this work, we study how to knot plastic bags that are randomly dropped from the air with a dual-arm robot based on image input. The complex initial configuration and terrible material properties of plastic bags pose challenges to reliable perception and planning. Directly knotting it from random initial states is difficult. To tackle this problem, we propose Iterative Interactive Modeling (IIM) to first adjust the plastic bag to a standing pose with imitation learning to establish a high-confidence keypoint skeleton model, then perform a set of learned motion primitives to knot it. We leverage spatial action maps to accomplish the iterative pick-and-place action and a graph convolutional network to evaluate the adjusted pose during the IIM process. In experiments, we achieve an 85.0% success rate in knotting 4 different plastic bags, including one with no demonstration.

Cite this Paper

BibTeX
@InProceedings{pmlr-v205-gao23a, title = {Iterative Interactive Modeling for Knotting Plastic Bags}, author = {Gao, Chongkai and Li, Zekun and Gao, Haichuan and Chen, Feng}, booktitle = {Proceedings of The 6th Conference on Robot Learning}, pages = {571--582}, year = {2023}, editor = {Liu, Karen and Kulic, Dana and Ichnowski, Jeff}, volume = {205}, series = {Proceedings of Machine Learning Research}, month = {14--18 Dec}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v205/gao23a/gao23a.pdf}, url = {https://proceedings.mlr.press/v205/gao23a.html}, abstract = {Deformable object manipulation has great research significance for the robotic community and numerous applications in daily life. In this work, we study how to knot plastic bags that are randomly dropped from the air with a dual-arm robot based on image input. The complex initial configuration and terrible material properties of plastic bags pose challenges to reliable perception and planning. Directly knotting it from random initial states is difficult. To tackle this problem, we propose Iterative Interactive Modeling (IIM) to first adjust the plastic bag to a standing pose with imitation learning to establish a high-confidence keypoint skeleton model, then perform a set of learned motion primitives to knot it. We leverage spatial action maps to accomplish the iterative pick-and-place action and a graph convolutional network to evaluate the adjusted pose during the IIM process. In experiments, we achieve an 85.0% success rate in knotting 4 different plastic bags, including one with no demonstration.} }
Endnote
%0 Conference Paper %T Iterative Interactive Modeling for Knotting Plastic Bags %A Chongkai Gao %A Zekun Li %A Haichuan Gao %A Feng Chen %B Proceedings of The 6th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2023 %E Karen Liu %E Dana Kulic %E Jeff Ichnowski %F pmlr-v205-gao23a %I PMLR %P 571--582 %U https://proceedings.mlr.press/v205/gao23a.html %V 205 %X Deformable object manipulation has great research significance for the robotic community and numerous applications in daily life. In this work, we study how to knot plastic bags that are randomly dropped from the air with a dual-arm robot based on image input. The complex initial configuration and terrible material properties of plastic bags pose challenges to reliable perception and planning. Directly knotting it from random initial states is difficult. To tackle this problem, we propose Iterative Interactive Modeling (IIM) to first adjust the plastic bag to a standing pose with imitation learning to establish a high-confidence keypoint skeleton model, then perform a set of learned motion primitives to knot it. We leverage spatial action maps to accomplish the iterative pick-and-place action and a graph convolutional network to evaluate the adjusted pose during the IIM process. In experiments, we achieve an 85.0% success rate in knotting 4 different plastic bags, including one with no demonstration.
APA
Gao, C., Li, Z., Gao, H. & Chen, F.. (2023). Iterative Interactive Modeling for Knotting Plastic Bags. Proceedings of The 6th Conference on Robot Learning, in Proceedings of Machine Learning Research 205:571-582 Available from https://proceedings.mlr.press/v205/gao23a.html.

Related Material