Vision-based Uneven BEV Representation Learning with Polar Rasterization and Surface Estimation

Zhi Liu, Shaoyu Chen, Xiaojie Guo, Xinggang Wang, Tianheng Cheng, Hongmei Zhu, Qian Zhang, Wenyu Liu, Yi Zhang
Proceedings of The 6th Conference on Robot Learning, PMLR 205:437-446, 2023.

Abstract

In this work, we propose PolarBEV for vision-based uneven BEV representation learning. To adapt to the foreshortening effect of camera imaging, we rasterize the BEV space both angularly and radially, and introduce polar embedding decomposition to model the associations among polar grids. Polar grids are rearranged to an array-like regular representation for efficient processing. Besides, to determine the 2D-to-3D correspondence, we iteratively update the BEV surface based on a hypothetical plane, and adopt height-based feature transformation. PolarBEV keeps real-time inference speed on a single 2080Ti GPU, and outperforms other methods for both BEV semantic segmentation and BEV instance segmentation. Thorough ablations are presented to validate the design. The code will be released for facilitating further research.

Cite this Paper

BibTeX
@InProceedings{pmlr-v205-liu23a, title = {Vision-based Uneven BEV Representation Learning with Polar Rasterization and Surface Estimation}, author = {Liu, Zhi and Chen, Shaoyu and Guo, Xiaojie and Wang, Xinggang and Cheng, Tianheng and Zhu, Hongmei and Zhang, Qian and Liu, Wenyu and Zhang, Yi}, booktitle = {Proceedings of The 6th Conference on Robot Learning}, pages = {437--446}, year = {2023}, editor = {Liu, Karen and Kulic, Dana and Ichnowski, Jeff}, volume = {205}, series = {Proceedings of Machine Learning Research}, month = {14--18 Dec}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v205/liu23a/liu23a.pdf}, url = {https://proceedings.mlr.press/v205/liu23a.html}, abstract = {In this work, we propose PolarBEV for vision-based uneven BEV representation learning. To adapt to the foreshortening effect of camera imaging, we rasterize the BEV space both angularly and radially, and introduce polar embedding decomposition to model the associations among polar grids. Polar grids are rearranged to an array-like regular representation for efficient processing. Besides, to determine the 2D-to-3D correspondence, we iteratively update the BEV surface based on a hypothetical plane, and adopt height-based feature transformation. PolarBEV keeps real-time inference speed on a single 2080Ti GPU, and outperforms other methods for both BEV semantic segmentation and BEV instance segmentation. Thorough ablations are presented to validate the design. The code will be released for facilitating further research.} }
Endnote
%0 Conference Paper %T Vision-based Uneven BEV Representation Learning with Polar Rasterization and Surface Estimation %A Zhi Liu %A Shaoyu Chen %A Xiaojie Guo %A Xinggang Wang %A Tianheng Cheng %A Hongmei Zhu %A Qian Zhang %A Wenyu Liu %A Yi Zhang %B Proceedings of The 6th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2023 %E Karen Liu %E Dana Kulic %E Jeff Ichnowski %F pmlr-v205-liu23a %I PMLR %P 437--446 %U https://proceedings.mlr.press/v205/liu23a.html %V 205 %X In this work, we propose PolarBEV for vision-based uneven BEV representation learning. To adapt to the foreshortening effect of camera imaging, we rasterize the BEV space both angularly and radially, and introduce polar embedding decomposition to model the associations among polar grids. Polar grids are rearranged to an array-like regular representation for efficient processing. Besides, to determine the 2D-to-3D correspondence, we iteratively update the BEV surface based on a hypothetical plane, and adopt height-based feature transformation. PolarBEV keeps real-time inference speed on a single 2080Ti GPU, and outperforms other methods for both BEV semantic segmentation and BEV instance segmentation. Thorough ablations are presented to validate the design. The code will be released for facilitating further research.
APA
Liu, Z., Chen, S., Guo, X., Wang, X., Cheng, T., Zhu, H., Zhang, Q., Liu, W. & Zhang, Y.. (2023). Vision-based Uneven BEV Representation Learning with Polar Rasterization and Surface Estimation. Proceedings of The 6th Conference on Robot Learning, in Proceedings of Machine Learning Research 205:437-446 Available from https://proceedings.mlr.press/v205/liu23a.html.

Related Material