StrObe: Streaming Object Detection from LiDAR Packets

Davi Frossard, Shun Da Suo, Sergio Casas, James Tu, Raquel Urtasun
Proceedings of the 2020 Conference on Robot Learning, PMLR 155:1174-1183, 2021.

Abstract

Many modern robotics systems employ LiDAR as their main sensing modality due to its geometrical richness. Rolling shutter LIDARs are particularly common, in which a sweep is built through the accumulation of points over an entire revolution of the sensor, thus producing a 360 point cloud of the scene. Modern perception algorithms wait for the full sweep to be built before processing the data, which introduces an additional latency of up to 100ms. As a consequence, by the time an output is produced, it no longer accurately reflects the state of the world. This poses a big challenge, as robotics applications require minimal reaction times, such that maneuvers can be quickly planned in the event of a safety-critical situation. In this paper we propose StrObe, a novel approach that minimizes latency by ingesting packets of LiDAR and emitting a stream of detections without waiting for the full sweep to be built. StrObe reuses computations from previous points and iteratively updates the spatial memory of the scene as new evidence comes in, resulting in latency reduced accurate perception. We demonstrate the effectiveness of our approach on a large scale dataset, showing that our approach far outperforms the state-of-the-art when latency is taken into account while still matching the performance in the traditional setting.

Cite this Paper

BibTeX
@InProceedings{pmlr-v155-frossard21a, title = {StrObe: Streaming Object Detection from LiDAR Packets}, author = {Frossard, Davi and Suo, Shun Da and Casas, Sergio and Tu, James and Urtasun, Raquel}, booktitle = {Proceedings of the 2020 Conference on Robot Learning}, pages = {1174--1183}, year = {2021}, editor = {Kober, Jens and Ramos, Fabio and Tomlin, Claire}, volume = {155}, series = {Proceedings of Machine Learning Research}, month = {16--18 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v155/frossard21a/frossard21a.pdf}, url = {https://proceedings.mlr.press/v155/frossard21a.html}, abstract = {Many modern robotics systems employ LiDAR as their main sensing modality due to its geometrical richness. Rolling shutter LIDARs are particularly common, in which a sweep is built through the accumulation of points over an entire revolution of the sensor, thus producing a 360 point cloud of the scene. Modern perception algorithms wait for the full sweep to be built before processing the data, which introduces an additional latency of up to 100ms. As a consequence, by the time an output is produced, it no longer accurately reflects the state of the world. This poses a big challenge, as robotics applications require minimal reaction times, such that maneuvers can be quickly planned in the event of a safety-critical situation. In this paper we propose StrObe, a novel approach that minimizes latency by ingesting packets of LiDAR and emitting a stream of detections without waiting for the full sweep to be built. StrObe reuses computations from previous points and iteratively updates the spatial memory of the scene as new evidence comes in, resulting in latency reduced accurate perception. We demonstrate the effectiveness of our approach on a large scale dataset, showing that our approach far outperforms the state-of-the-art when latency is taken into account while still matching the performance in the traditional setting.} }
Endnote
%0 Conference Paper %T StrObe: Streaming Object Detection from LiDAR Packets %A Davi Frossard %A Shun Da Suo %A Sergio Casas %A James Tu %A Raquel Urtasun %B Proceedings of the 2020 Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2021 %E Jens Kober %E Fabio Ramos %E Claire Tomlin %F pmlr-v155-frossard21a %I PMLR %P 1174--1183 %U https://proceedings.mlr.press/v155/frossard21a.html %V 155 %X Many modern robotics systems employ LiDAR as their main sensing modality due to its geometrical richness. Rolling shutter LIDARs are particularly common, in which a sweep is built through the accumulation of points over an entire revolution of the sensor, thus producing a 360 point cloud of the scene. Modern perception algorithms wait for the full sweep to be built before processing the data, which introduces an additional latency of up to 100ms. As a consequence, by the time an output is produced, it no longer accurately reflects the state of the world. This poses a big challenge, as robotics applications require minimal reaction times, such that maneuvers can be quickly planned in the event of a safety-critical situation. In this paper we propose StrObe, a novel approach that minimizes latency by ingesting packets of LiDAR and emitting a stream of detections without waiting for the full sweep to be built. StrObe reuses computations from previous points and iteratively updates the spatial memory of the scene as new evidence comes in, resulting in latency reduced accurate perception. We demonstrate the effectiveness of our approach on a large scale dataset, showing that our approach far outperforms the state-of-the-art when latency is taken into account while still matching the performance in the traditional setting.
APA
Frossard, D., Suo, S.D., Casas, S., Tu, J. & Urtasun, R.. (2021). StrObe: Streaming Object Detection from LiDAR Packets. Proceedings of the 2020 Conference on Robot Learning, in Proceedings of Machine Learning Research 155:1174-1183 Available from https://proceedings.mlr.press/v155/frossard21a.html.

Related Material