Learning Road Scene-level Representations via Semantic Region Prediction

Zihao Xiao, Alan Yuille, Yi-Ting Chen
Proceedings of The 6th Conference on Robot Learning, PMLR 205:1882-1892, 2023.

Abstract

In this work, we tackle two vital tasks in automated driving systems, i.e., driver intent prediction and risk object identification from egocentric images. Mainly, we investigate the question: what would be good road scene-level representations for these two tasks? We contend that a scene-level representation must capture higher-level semantic and geometric representations of traffic scenes around ego-vehicle while performing actions to their destinations. To this end, we introduce the representation of semantic regions, which are areas where ego-vehicles visit while taking an afforded action (e.g., left-turn at 4-way intersections). We propose to learn scene-level representations via a novel semantic region prediction task and an automatic semantic region labeling algorithm. Extensive evaluations are conducted on the HDD and nuScenes datasets, and the learned representations lead to state-of-the-art performance for driver intention prediction and risk object identification.

Cite this Paper

BibTeX
@InProceedings{pmlr-v205-xiao23b, title = {Learning Road Scene-level Representations via Semantic Region Prediction}, author = {Xiao, Zihao and Yuille, Alan and Chen, Yi-Ting}, booktitle = {Proceedings of The 6th Conference on Robot Learning}, pages = {1882--1892}, year = {2023}, editor = {Liu, Karen and Kulic, Dana and Ichnowski, Jeff}, volume = {205}, series = {Proceedings of Machine Learning Research}, month = {14--18 Dec}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v205/xiao23b/xiao23b.pdf}, url = {https://proceedings.mlr.press/v205/xiao23b.html}, abstract = {In this work, we tackle two vital tasks in automated driving systems, i.e., driver intent prediction and risk object identification from egocentric images. Mainly, we investigate the question: what would be good road scene-level representations for these two tasks? We contend that a scene-level representation must capture higher-level semantic and geometric representations of traffic scenes around ego-vehicle while performing actions to their destinations. To this end, we introduce the representation of semantic regions, which are areas where ego-vehicles visit while taking an afforded action (e.g., left-turn at 4-way intersections). We propose to learn scene-level representations via a novel semantic region prediction task and an automatic semantic region labeling algorithm. Extensive evaluations are conducted on the HDD and nuScenes datasets, and the learned representations lead to state-of-the-art performance for driver intention prediction and risk object identification. } }
Endnote
%0 Conference Paper %T Learning Road Scene-level Representations via Semantic Region Prediction %A Zihao Xiao %A Alan Yuille %A Yi-Ting Chen %B Proceedings of The 6th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2023 %E Karen Liu %E Dana Kulic %E Jeff Ichnowski %F pmlr-v205-xiao23b %I PMLR %P 1882--1892 %U https://proceedings.mlr.press/v205/xiao23b.html %V 205 %X In this work, we tackle two vital tasks in automated driving systems, i.e., driver intent prediction and risk object identification from egocentric images. Mainly, we investigate the question: what would be good road scene-level representations for these two tasks? We contend that a scene-level representation must capture higher-level semantic and geometric representations of traffic scenes around ego-vehicle while performing actions to their destinations. To this end, we introduce the representation of semantic regions, which are areas where ego-vehicles visit while taking an afforded action (e.g., left-turn at 4-way intersections). We propose to learn scene-level representations via a novel semantic region prediction task and an automatic semantic region labeling algorithm. Extensive evaluations are conducted on the HDD and nuScenes datasets, and the learned representations lead to state-of-the-art performance for driver intention prediction and risk object identification.
APA
Xiao, Z., Yuille, A. & Chen, Y.. (2023). Learning Road Scene-level Representations via Semantic Region Prediction. Proceedings of The 6th Conference on Robot Learning, in Proceedings of Machine Learning Research 205:1882-1892 Available from https://proceedings.mlr.press/v205/xiao23b.html.

Related Material