S2TNet: Spatio-Temporal Transformer Networks for Trajectory Prediction in Autonomous Driving

Weihuang Chen, Fangfang Wang, Hongbin Sun
Proceedings of The 13th Asian Conference on Machine Learning, PMLR 157:454-469, 2021.

Abstract

To safely and rationally participate in dense and heterogeneous traffic, autonomous vehicles require to sufficiently analyze the motion patterns of surrounding traffic-agents and accurately predict their future trajectories. This is challenging because the trajectories of traffic-agents are not only influenced by the traffic-agents themselves but also by spatial interaction with each other. Previous methods usually rely on the sequential step-by-step processing of Long Short-Term Memory networks (LSTMs) and merely extract the interactions between spatial neighbors for single type traffic-agents. We propose the Spatio-Temporal Transformer Networks (S2TNet), which models the spatio-temporal interactions by spatio-temporal Transformer and deals with the temporel sequences by temporal Transformer. We input additional category, shape and heading information into our networks to handle the heterogeneity of traffic-agents. The proposed methods outperforms state-of-the-art methods on ApolloScape Trajectory dataset by more than 7% on both the weighted sum of Average and Final Displacement Error.

Cite this Paper

BibTeX
@InProceedings{pmlr-v157-chen21a, title = {S2TNet: Spatio-Temporal Transformer Networks for Trajectory Prediction in Autonomous Driving}, author = {Chen, Weihuang and Wang, Fangfang and Sun, Hongbin}, booktitle = {Proceedings of The 13th Asian Conference on Machine Learning}, pages = {454--469}, year = {2021}, editor = {Balasubramanian, Vineeth N. and Tsang, Ivor}, volume = {157}, series = {Proceedings of Machine Learning Research}, month = {17--19 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v157/chen21a/chen21a.pdf}, url = {https://proceedings.mlr.press/v157/chen21a.html}, abstract = {To safely and rationally participate in dense and heterogeneous traffic, autonomous vehicles require to sufficiently analyze the motion patterns of surrounding traffic-agents and accurately predict their future trajectories. This is challenging because the trajectories of traffic-agents are not only influenced by the traffic-agents themselves but also by spatial interaction with each other. Previous methods usually rely on the sequential step-by-step processing of Long Short-Term Memory networks (LSTMs) and merely extract the interactions between spatial neighbors for single type traffic-agents. We propose the Spatio-Temporal Transformer Networks (S2TNet), which models the spatio-temporal interactions by spatio-temporal Transformer and deals with the temporel sequences by temporal Transformer. We input additional category, shape and heading information into our networks to handle the heterogeneity of traffic-agents. The proposed methods outperforms state-of-the-art methods on ApolloScape Trajectory dataset by more than 7% on both the weighted sum of Average and Final Displacement Error.} }
Endnote
%0 Conference Paper %T S2TNet: Spatio-Temporal Transformer Networks for Trajectory Prediction in Autonomous Driving %A Weihuang Chen %A Fangfang Wang %A Hongbin Sun %B Proceedings of The 13th Asian Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Vineeth N. Balasubramanian %E Ivor Tsang %F pmlr-v157-chen21a %I PMLR %P 454--469 %U https://proceedings.mlr.press/v157/chen21a.html %V 157 %X To safely and rationally participate in dense and heterogeneous traffic, autonomous vehicles require to sufficiently analyze the motion patterns of surrounding traffic-agents and accurately predict their future trajectories. This is challenging because the trajectories of traffic-agents are not only influenced by the traffic-agents themselves but also by spatial interaction with each other. Previous methods usually rely on the sequential step-by-step processing of Long Short-Term Memory networks (LSTMs) and merely extract the interactions between spatial neighbors for single type traffic-agents. We propose the Spatio-Temporal Transformer Networks (S2TNet), which models the spatio-temporal interactions by spatio-temporal Transformer and deals with the temporel sequences by temporal Transformer. We input additional category, shape and heading information into our networks to handle the heterogeneity of traffic-agents. The proposed methods outperforms state-of-the-art methods on ApolloScape Trajectory dataset by more than 7% on both the weighted sum of Average and Final Displacement Error.
APA
Chen, W., Wang, F. & Sun, H.. (2021). S2TNet: Spatio-Temporal Transformer Networks for Trajectory Prediction in Autonomous Driving. Proceedings of The 13th Asian Conference on Machine Learning, in Proceedings of Machine Learning Research 157:454-469 Available from https://proceedings.mlr.press/v157/chen21a.html.

Related Material