Two Stream Networks for Self-Supervised Ego-Motion Estimation

Rares Ambrus, Vitor Guizilini, Jie Li, Sudeep Pillai Adrien Gaidon
Proceedings of the Conference on Robot Learning, PMLR 100:1052-1061, 2020.

Abstract

Learning depth and camera ego-motion from raw unlabeled RGB video streams is seeing exciting progress through self-supervision from strong geometric cues. To leverage not only appearance but also scene geometry, we propose a novel self-supervised two-stream network using RGB and inferred depth information for accurate visual odometry. In addition, we introduce a sparsity-inducing data augmentation policy for ego-motion learning that effectively regularizes the pose network to enable stronger generalization performance. As a result, we show that our proposed two-stream pose network achieves state-of-the-art results among learning-based methods on the KITTI odometry benchmark, and is especially suited for self-supervision at scale. Our experiments on a large-scale urban driving dataset of 1 million frames indicate that the performance of our proposed architecture does indeed scale progressively with more data.

Cite this Paper

BibTeX
@InProceedings{pmlr-v100-ambrus20a, title = {Two Stream Networks for Self-Supervised Ego-Motion Estimation}, author = {Ambrus, Rares and Guizilini, Vitor and Li, Jie and Gaidon, Sudeep Pillai Adrien}, booktitle = {Proceedings of the Conference on Robot Learning}, pages = {1052--1061}, year = {2020}, editor = {Kaelbling, Leslie Pack and Kragic, Danica and Sugiura, Komei}, volume = {100}, series = {Proceedings of Machine Learning Research}, month = {30 Oct--01 Nov}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v100/ambrus20a/ambrus20a.pdf}, url = {https://proceedings.mlr.press/v100/ambrus20a.html}, abstract = {Learning depth and camera ego-motion from raw unlabeled RGB video streams is seeing exciting progress through self-supervision from strong geometric cues. To leverage not only appearance but also scene geometry, we propose a novel self-supervised two-stream network using RGB and inferred depth information for accurate visual odometry. In addition, we introduce a sparsity-inducing data augmentation policy for ego-motion learning that effectively regularizes the pose network to enable stronger generalization performance. As a result, we show that our proposed two-stream pose network achieves state-of-the-art results among learning-based methods on the KITTI odometry benchmark, and is especially suited for self-supervision at scale. Our experiments on a large-scale urban driving dataset of 1 million frames indicate that the performance of our proposed architecture does indeed scale progressively with more data.} }
Endnote
%0 Conference Paper %T Two Stream Networks for Self-Supervised Ego-Motion Estimation %A Rares Ambrus %A Vitor Guizilini %A Jie Li %A Sudeep Pillai Adrien Gaidon %B Proceedings of the Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2020 %E Leslie Pack Kaelbling %E Danica Kragic %E Komei Sugiura %F pmlr-v100-ambrus20a %I PMLR %P 1052--1061 %U https://proceedings.mlr.press/v100/ambrus20a.html %V 100 %X Learning depth and camera ego-motion from raw unlabeled RGB video streams is seeing exciting progress through self-supervision from strong geometric cues. To leverage not only appearance but also scene geometry, we propose a novel self-supervised two-stream network using RGB and inferred depth information for accurate visual odometry. In addition, we introduce a sparsity-inducing data augmentation policy for ego-motion learning that effectively regularizes the pose network to enable stronger generalization performance. As a result, we show that our proposed two-stream pose network achieves state-of-the-art results among learning-based methods on the KITTI odometry benchmark, and is especially suited for self-supervision at scale. Our experiments on a large-scale urban driving dataset of 1 million frames indicate that the performance of our proposed architecture does indeed scale progressively with more data.
APA
Ambrus, R., Guizilini, V., Li, J. & Gaidon, S.P.A.. (2020). Two Stream Networks for Self-Supervised Ego-Motion Estimation. Proceedings of the Conference on Robot Learning, in Proceedings of Machine Learning Research 100:1052-1061 Available from https://proceedings.mlr.press/v100/ambrus20a.html.

Related Material