Learning to Communicate and Correct Pose Errors

Nicholas Vadivelu, Mengye Ren, James Tu, Jingkang Wang, Raquel Urtasun
Proceedings of the 2020 Conference on Robot Learning, PMLR 155:1195-1210, 2021.

Abstract

Learned communication makes multi-agent systems more effective by aggregating distributed information. However, it also exposes individual agents to the threat of erroneous messages they receive. In this paper, we study the setting proposed in V2VNet, where nearby self-driving vehicles jointly perform object detection and motion forecasting in a cooperative manner. Despite a huge performance boost when the agents solve the task together, the gain is quickly diminished in the presence of pose noise since the communication relies on spatial transformations. Hence, we propose a novel neural reasoning framework that learns to communicate, to estimate errors, and finally, to reach a consensus about those errors. Experiments confirm that our proposed framework significantly improves the robustness of multi-agent self-driving perception systems under realistic and severe localization noise.

Cite this Paper

BibTeX
@InProceedings{pmlr-v155-vadivelu21a, title = {Learning to Communicate and Correct Pose Errors}, author = {Vadivelu, Nicholas and Ren, Mengye and Tu, James and Wang, Jingkang and Urtasun, Raquel}, booktitle = {Proceedings of the 2020 Conference on Robot Learning}, pages = {1195--1210}, year = {2021}, editor = {Kober, Jens and Ramos, Fabio and Tomlin, Claire}, volume = {155}, series = {Proceedings of Machine Learning Research}, month = {16--18 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v155/vadivelu21a/vadivelu21a.pdf}, url = {https://proceedings.mlr.press/v155/vadivelu21a.html}, abstract = {Learned communication makes multi-agent systems more effective by aggregating distributed information. However, it also exposes individual agents to the threat of erroneous messages they receive. In this paper, we study the setting proposed in V2VNet, where nearby self-driving vehicles jointly perform object detection and motion forecasting in a cooperative manner. Despite a huge performance boost when the agents solve the task together, the gain is quickly diminished in the presence of pose noise since the communication relies on spatial transformations. Hence, we propose a novel neural reasoning framework that learns to communicate, to estimate errors, and finally, to reach a consensus about those errors. Experiments confirm that our proposed framework significantly improves the robustness of multi-agent self-driving perception systems under realistic and severe localization noise.} }
Endnote
%0 Conference Paper %T Learning to Communicate and Correct Pose Errors %A Nicholas Vadivelu %A Mengye Ren %A James Tu %A Jingkang Wang %A Raquel Urtasun %B Proceedings of the 2020 Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2021 %E Jens Kober %E Fabio Ramos %E Claire Tomlin %F pmlr-v155-vadivelu21a %I PMLR %P 1195--1210 %U https://proceedings.mlr.press/v155/vadivelu21a.html %V 155 %X Learned communication makes multi-agent systems more effective by aggregating distributed information. However, it also exposes individual agents to the threat of erroneous messages they receive. In this paper, we study the setting proposed in V2VNet, where nearby self-driving vehicles jointly perform object detection and motion forecasting in a cooperative manner. Despite a huge performance boost when the agents solve the task together, the gain is quickly diminished in the presence of pose noise since the communication relies on spatial transformations. Hence, we propose a novel neural reasoning framework that learns to communicate, to estimate errors, and finally, to reach a consensus about those errors. Experiments confirm that our proposed framework significantly improves the robustness of multi-agent self-driving perception systems under realistic and severe localization noise.
APA
Vadivelu, N., Ren, M., Tu, J., Wang, J. & Urtasun, R.. (2021). Learning to Communicate and Correct Pose Errors. Proceedings of the 2020 Conference on Robot Learning, in Proceedings of Machine Learning Research 155:1195-1210 Available from https://proceedings.mlr.press/v155/vadivelu21a.html.

Related Material