Probabilistic and Geometric Depth: Detecting Objects in Perspective

Tai Wang, Xinge ZHU, Jiangmiao Pang, Dahua Lin
Proceedings of the 5th Conference on Robot Learning, PMLR 164:1475-1485, 2022.

Abstract

3D object detection is an important capability needed in various practical applications such as driver assistance systems. Monocular 3D detection, a representative general setting among image-based approaches, provides a more economical solution than conventional settings relying on LiDARs but still yields unsatisfactory results. This paper first presents a systematic study on this problem. We observe that the current monocular 3D detection can be simplified as an instance depth estimation problem: The inaccurate instance depth blocks all the other 3D attribute predictions from improving the overall detection performance. Moreover, recent methods directly estimate the depth based on isolated instances or pixels while ignoring the geometric relations across different objects. To this end, we construct geometric relation graphs across predicted objects and use the graph to facilitate depth estimation. As the preliminary depth estimation of each instance is usually inaccurate in this ill-posed setting, we incorporate a probabilistic representation to capture the uncertainty. It provides an important indicator to identify confident predictions and further guide the depth propagation. Despite the simplicity of the basic idea, our method, PGD, obtains significant improvements on KITTI and nuScenes benchmarks, achieving 1st place out of all monocular vision-only methods while still maintaining real-time efficiency. Code and models will be released at https://github.com/open-mmlab/mmdetection3d.

Cite this Paper

BibTeX
@InProceedings{pmlr-v164-wang22i, title = {Probabilistic and Geometric Depth: Detecting Objects in Perspective}, author = {Wang, Tai and ZHU, Xinge and Pang, Jiangmiao and Lin, Dahua}, booktitle = {Proceedings of the 5th Conference on Robot Learning}, pages = {1475--1485}, year = {2022}, editor = {Faust, Aleksandra and Hsu, David and Neumann, Gerhard}, volume = {164}, series = {Proceedings of Machine Learning Research}, month = {08--11 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v164/wang22i/wang22i.pdf}, url = {https://proceedings.mlr.press/v164/wang22i.html}, abstract = {3D object detection is an important capability needed in various practical applications such as driver assistance systems. Monocular 3D detection, a representative general setting among image-based approaches, provides a more economical solution than conventional settings relying on LiDARs but still yields unsatisfactory results. This paper first presents a systematic study on this problem. We observe that the current monocular 3D detection can be simplified as an instance depth estimation problem: The inaccurate instance depth blocks all the other 3D attribute predictions from improving the overall detection performance. Moreover, recent methods directly estimate the depth based on isolated instances or pixels while ignoring the geometric relations across different objects. To this end, we construct geometric relation graphs across predicted objects and use the graph to facilitate depth estimation. As the preliminary depth estimation of each instance is usually inaccurate in this ill-posed setting, we incorporate a probabilistic representation to capture the uncertainty. It provides an important indicator to identify confident predictions and further guide the depth propagation. Despite the simplicity of the basic idea, our method, PGD, obtains significant improvements on KITTI and nuScenes benchmarks, achieving 1st place out of all monocular vision-only methods while still maintaining real-time efficiency. Code and models will be released at https://github.com/open-mmlab/mmdetection3d.} }
Endnote
%0 Conference Paper %T Probabilistic and Geometric Depth: Detecting Objects in Perspective %A Tai Wang %A Xinge ZHU %A Jiangmiao Pang %A Dahua Lin %B Proceedings of the 5th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2022 %E Aleksandra Faust %E David Hsu %E Gerhard Neumann %F pmlr-v164-wang22i %I PMLR %P 1475--1485 %U https://proceedings.mlr.press/v164/wang22i.html %V 164 %X 3D object detection is an important capability needed in various practical applications such as driver assistance systems. Monocular 3D detection, a representative general setting among image-based approaches, provides a more economical solution than conventional settings relying on LiDARs but still yields unsatisfactory results. This paper first presents a systematic study on this problem. We observe that the current monocular 3D detection can be simplified as an instance depth estimation problem: The inaccurate instance depth blocks all the other 3D attribute predictions from improving the overall detection performance. Moreover, recent methods directly estimate the depth based on isolated instances or pixels while ignoring the geometric relations across different objects. To this end, we construct geometric relation graphs across predicted objects and use the graph to facilitate depth estimation. As the preliminary depth estimation of each instance is usually inaccurate in this ill-posed setting, we incorporate a probabilistic representation to capture the uncertainty. It provides an important indicator to identify confident predictions and further guide the depth propagation. Despite the simplicity of the basic idea, our method, PGD, obtains significant improvements on KITTI and nuScenes benchmarks, achieving 1st place out of all monocular vision-only methods while still maintaining real-time efficiency. Code and models will be released at https://github.com/open-mmlab/mmdetection3d.
APA
Wang, T., ZHU, X., Pang, J. & Lin, D.. (2022). Probabilistic and Geometric Depth: Detecting Objects in Perspective. Proceedings of the 5th Conference on Robot Learning, in Proceedings of Machine Learning Research 164:1475-1485 Available from https://proceedings.mlr.press/v164/wang22i.html.

Related Material