Affordance-Driven Next-Best-View Planning for Robotic Grasping

Xuechao Zhang, Dong Wang, Sun Han, Weichuang Li, Bin Zhao, Zhigang Wang, Xiaoming Duan, Chongrong Fang, Xuelong Li, Jianping He
Proceedings of The 7th Conference on Robot Learning, PMLR 229:2849-2862, 2023.

Abstract

Grasping occluded objects in cluttered environments is an essential component in complex robotic manipulation tasks. In this paper, we introduce an AffordanCE-driven Next-Best-View planning policy (ACE-NBV) that tries to find a feasible grasp for target object via continuously observing scenes from new viewpoints. This policy is motivated by the observation that the grasp affordances of an occluded object can be better-measured under the view when the view-direction are the same as the grasp view. Specifically, our method leverages the paradigm of novel view imagery to predict the grasps affordances under previously unobserved view, and select next observation view based on the highest imagined grasp quality of the target object. The experimental results in simulation and on a real robot demonstrate the effectiveness of the proposed affordance-driven next-best-view planning policy. Project page: https://sszxc.net/ace-nbv/.

Cite this Paper

BibTeX
@InProceedings{pmlr-v229-zhang23i, title = {Affordance-Driven Next-Best-View Planning for Robotic Grasping}, author = {Zhang, Xuechao and Wang, Dong and Han, Sun and Li, Weichuang and Zhao, Bin and Wang, Zhigang and Duan, Xiaoming and Fang, Chongrong and Li, Xuelong and He, Jianping}, booktitle = {Proceedings of The 7th Conference on Robot Learning}, pages = {2849--2862}, year = {2023}, editor = {Tan, Jie and Toussaint, Marc and Darvish, Kourosh}, volume = {229}, series = {Proceedings of Machine Learning Research}, month = {06--09 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v229/zhang23i/zhang23i.pdf}, url = {https://proceedings.mlr.press/v229/zhang23i.html}, abstract = {Grasping occluded objects in cluttered environments is an essential component in complex robotic manipulation tasks. In this paper, we introduce an AffordanCE-driven Next-Best-View planning policy (ACE-NBV) that tries to find a feasible grasp for target object via continuously observing scenes from new viewpoints. This policy is motivated by the observation that the grasp affordances of an occluded object can be better-measured under the view when the view-direction are the same as the grasp view. Specifically, our method leverages the paradigm of novel view imagery to predict the grasps affordances under previously unobserved view, and select next observation view based on the highest imagined grasp quality of the target object. The experimental results in simulation and on a real robot demonstrate the effectiveness of the proposed affordance-driven next-best-view planning policy. Project page: https://sszxc.net/ace-nbv/.} }
Endnote
%0 Conference Paper %T Affordance-Driven Next-Best-View Planning for Robotic Grasping %A Xuechao Zhang %A Dong Wang %A Sun Han %A Weichuang Li %A Bin Zhao %A Zhigang Wang %A Xiaoming Duan %A Chongrong Fang %A Xuelong Li %A Jianping He %B Proceedings of The 7th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2023 %E Jie Tan %E Marc Toussaint %E Kourosh Darvish %F pmlr-v229-zhang23i %I PMLR %P 2849--2862 %U https://proceedings.mlr.press/v229/zhang23i.html %V 229 %X Grasping occluded objects in cluttered environments is an essential component in complex robotic manipulation tasks. In this paper, we introduce an AffordanCE-driven Next-Best-View planning policy (ACE-NBV) that tries to find a feasible grasp for target object via continuously observing scenes from new viewpoints. This policy is motivated by the observation that the grasp affordances of an occluded object can be better-measured under the view when the view-direction are the same as the grasp view. Specifically, our method leverages the paradigm of novel view imagery to predict the grasps affordances under previously unobserved view, and select next observation view based on the highest imagined grasp quality of the target object. The experimental results in simulation and on a real robot demonstrate the effectiveness of the proposed affordance-driven next-best-view planning policy. Project page: https://sszxc.net/ace-nbv/.
APA
Zhang, X., Wang, D., Han, S., Li, W., Zhao, B., Wang, Z., Duan, X., Fang, C., Li, X. & He, J.. (2023). Affordance-Driven Next-Best-View Planning for Robotic Grasping. Proceedings of The 7th Conference on Robot Learning, in Proceedings of Machine Learning Research 229:2849-2862 Available from https://proceedings.mlr.press/v229/zhang23i.html.

Related Material