Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformer

Hao Shao, Letian Wang, Ruobing Chen, Hongsheng Li, Yu Liu
Proceedings of The 6th Conference on Robot Learning, PMLR 205:726-737, 2023.

Abstract

Large-scale deployment of autonomous vehicles has been continually delayed due to safety concerns. On the one hand, comprehensive scene understanding is indispensable, a lack of which would result in vulnerability to rare but complex traffic situations, such as the sudden emergence of unknown objects. However, reasoning from a global context requires access to sensors of multiple types and adequate fusion of multi-modal sensor signals, which is difficult to achieve. On the other hand, the lack of interpretability in learning models also hampers the safety with unverifiable failure causes. In this paper, we propose a safety-enhanced autonomous driving framework, named Interpretable Sensor Fusion Transformer (InterFuser), to fully process and fuse information from multi-modal multi-view sensors for achieving comprehensive scene understanding and adversarial event detection. Besides, intermediate interpretable features are generated from our framework, which provide more semantics and are exploited to better constrain actions to be within the safe sets. We conducted extensive experiments on CARLA benchmarks, where our model outperforms prior methods, ranking the first on the public CARLA Leaderboard.

Cite this Paper

BibTeX
@InProceedings{pmlr-v205-shao23a, title = {Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformer}, author = {Shao, Hao and Wang, Letian and Chen, Ruobing and Li, Hongsheng and Liu, Yu}, booktitle = {Proceedings of The 6th Conference on Robot Learning}, pages = {726--737}, year = {2023}, editor = {Liu, Karen and Kulic, Dana and Ichnowski, Jeff}, volume = {205}, series = {Proceedings of Machine Learning Research}, month = {14--18 Dec}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v205/shao23a/shao23a.pdf}, url = {https://proceedings.mlr.press/v205/shao23a.html}, abstract = {Large-scale deployment of autonomous vehicles has been continually delayed due to safety concerns. On the one hand, comprehensive scene understanding is indispensable, a lack of which would result in vulnerability to rare but complex traffic situations, such as the sudden emergence of unknown objects. However, reasoning from a global context requires access to sensors of multiple types and adequate fusion of multi-modal sensor signals, which is difficult to achieve. On the other hand, the lack of interpretability in learning models also hampers the safety with unverifiable failure causes. In this paper, we propose a safety-enhanced autonomous driving framework, named Interpretable Sensor Fusion Transformer (InterFuser), to fully process and fuse information from multi-modal multi-view sensors for achieving comprehensive scene understanding and adversarial event detection. Besides, intermediate interpretable features are generated from our framework, which provide more semantics and are exploited to better constrain actions to be within the safe sets. We conducted extensive experiments on CARLA benchmarks, where our model outperforms prior methods, ranking the first on the public CARLA Leaderboard.} }
Endnote
%0 Conference Paper %T Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformer %A Hao Shao %A Letian Wang %A Ruobing Chen %A Hongsheng Li %A Yu Liu %B Proceedings of The 6th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2023 %E Karen Liu %E Dana Kulic %E Jeff Ichnowski %F pmlr-v205-shao23a %I PMLR %P 726--737 %U https://proceedings.mlr.press/v205/shao23a.html %V 205 %X Large-scale deployment of autonomous vehicles has been continually delayed due to safety concerns. On the one hand, comprehensive scene understanding is indispensable, a lack of which would result in vulnerability to rare but complex traffic situations, such as the sudden emergence of unknown objects. However, reasoning from a global context requires access to sensors of multiple types and adequate fusion of multi-modal sensor signals, which is difficult to achieve. On the other hand, the lack of interpretability in learning models also hampers the safety with unverifiable failure causes. In this paper, we propose a safety-enhanced autonomous driving framework, named Interpretable Sensor Fusion Transformer (InterFuser), to fully process and fuse information from multi-modal multi-view sensors for achieving comprehensive scene understanding and adversarial event detection. Besides, intermediate interpretable features are generated from our framework, which provide more semantics and are exploited to better constrain actions to be within the safe sets. We conducted extensive experiments on CARLA benchmarks, where our model outperforms prior methods, ranking the first on the public CARLA Leaderboard.
APA
Shao, H., Wang, L., Chen, R., Li, H. & Liu, Y.. (2023). Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformer. Proceedings of The 6th Conference on Robot Learning, in Proceedings of Machine Learning Research 205:726-737 Available from https://proceedings.mlr.press/v205/shao23a.html.

Related Material