JFP: Joint Future Prediction with Interactive Multi-Agent Modeling for Autonomous Driving

Wenjie Luo, Cheol Park, Andre Cornman, Benjamin Sapp, Dragomir Anguelov
Proceedings of The 6th Conference on Robot Learning, PMLR 205:1457-1467, 2023.

Abstract

We propose \textit{JFP}, a Joint Future Prediction model that can learn to generate accurate and consistent multi-agent future trajectories. For this task, many different methods have been proposed to capture social interactions in the encoding part of the model, however, considerably less focus has been placed on representing interactions in the decoder and output stages. As a result, the predicted trajectories are not necessarily consistent with each other, and often result in unrealistic trajectory overlaps. In contrast, we propose an end-to-end trainable model that learns directly the interaction between pairs of agents in a structured, graphical model formulation in order to generate consistent future trajectories. It sets new state-of-the-art results on Waymo Open Motion Dataset (WOMD) for the interactive setting. We also investigate a more complex multi-agent setting for both WOMD and a larger internal dataset, where our approach improves significantly on the trajectory overlap metrics while obtaining on-par or better performance on single-agent trajectory metrics.

Cite this Paper

BibTeX
@InProceedings{pmlr-v205-luo23a, title = {JFP: Joint Future Prediction with Interactive Multi-Agent Modeling for Autonomous Driving}, author = {Luo, Wenjie and Park, Cheol and Cornman, Andre and Sapp, Benjamin and Anguelov, Dragomir}, booktitle = {Proceedings of The 6th Conference on Robot Learning}, pages = {1457--1467}, year = {2023}, editor = {Liu, Karen and Kulic, Dana and Ichnowski, Jeff}, volume = {205}, series = {Proceedings of Machine Learning Research}, month = {14--18 Dec}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v205/luo23a/luo23a.pdf}, url = {https://proceedings.mlr.press/v205/luo23a.html}, abstract = {We propose \textit{JFP}, a Joint Future Prediction model that can learn to generate accurate and consistent multi-agent future trajectories. For this task, many different methods have been proposed to capture social interactions in the encoding part of the model, however, considerably less focus has been placed on representing interactions in the decoder and output stages. As a result, the predicted trajectories are not necessarily consistent with each other, and often result in unrealistic trajectory overlaps. In contrast, we propose an end-to-end trainable model that learns directly the interaction between pairs of agents in a structured, graphical model formulation in order to generate consistent future trajectories. It sets new state-of-the-art results on Waymo Open Motion Dataset (WOMD) for the interactive setting. We also investigate a more complex multi-agent setting for both WOMD and a larger internal dataset, where our approach improves significantly on the trajectory overlap metrics while obtaining on-par or better performance on single-agent trajectory metrics.} }
Endnote
%0 Conference Paper %T JFP: Joint Future Prediction with Interactive Multi-Agent Modeling for Autonomous Driving %A Wenjie Luo %A Cheol Park %A Andre Cornman %A Benjamin Sapp %A Dragomir Anguelov %B Proceedings of The 6th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2023 %E Karen Liu %E Dana Kulic %E Jeff Ichnowski %F pmlr-v205-luo23a %I PMLR %P 1457--1467 %U https://proceedings.mlr.press/v205/luo23a.html %V 205 %X We propose \textit{JFP}, a Joint Future Prediction model that can learn to generate accurate and consistent multi-agent future trajectories. For this task, many different methods have been proposed to capture social interactions in the encoding part of the model, however, considerably less focus has been placed on representing interactions in the decoder and output stages. As a result, the predicted trajectories are not necessarily consistent with each other, and often result in unrealistic trajectory overlaps. In contrast, we propose an end-to-end trainable model that learns directly the interaction between pairs of agents in a structured, graphical model formulation in order to generate consistent future trajectories. It sets new state-of-the-art results on Waymo Open Motion Dataset (WOMD) for the interactive setting. We also investigate a more complex multi-agent setting for both WOMD and a larger internal dataset, where our approach improves significantly on the trajectory overlap metrics while obtaining on-par or better performance on single-agent trajectory metrics.
APA
Luo, W., Park, C., Cornman, A., Sapp, B. & Anguelov, D.. (2023). JFP: Joint Future Prediction with Interactive Multi-Agent Modeling for Autonomous Driving. Proceedings of The 6th Conference on Robot Learning, in Proceedings of Machine Learning Research 205:1457-1467 Available from https://proceedings.mlr.press/v205/luo23a.html.

Related Material