Two Stream Networks for Self-Supervised Ego-Motion Estimation

Rares Ambrus, Vitor Guizilini, Jie Li, Sudeep Pillai Adrien Gaidon
; Proceedings of the Conference on Robot Learning, PMLR 100:1052-1061, 2020.

Abstract

Learning depth and camera ego-motion from raw unlabeled RGB video streams is seeing exciting progress through self-supervision from strong geometric cues. To leverage not only appearance but also scene geometry, we propose a novel self-supervised two-stream network using RGB and inferred depth information for accurate visual odometry. In addition, we introduce a sparsity-inducing data augmentation policy for ego-motion learning that effectively regularizes the pose network to enable stronger generalization performance. As a result, we show that our proposed two-stream pose network achieves state-of-the-art results among learning-based methods on the KITTI odometry benchmark, and is especially suited for self-supervision at scale. Our experiments on a large-scale urban driving dataset of 1 million frames indicate that the performance of our proposed architecture does indeed scale progressively with more data.

Cite this Paper


BibTeX
@InProceedings{pmlr-v100-ambrus20a, title = {Two Stream Networks for Self-Supervised Ego-Motion Estimation}, author = {Ambrus, Rares and Guizilini, Vitor and Li, Jie and Gaidon, Sudeep Pillai Adrien}, booktitle = {Proceedings of the Conference on Robot Learning}, pages = {1052--1061}, year = {2020}, editor = {Leslie Pack Kaelbling and Danica Kragic and Komei Sugiura}, volume = {100}, series = {Proceedings of Machine Learning Research}, address = {}, month = {30 Oct--01 Nov}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v100/ambrus20a/ambrus20a.pdf}, url = {http://proceedings.mlr.press/v100/ambrus20a.html}, abstract = {Learning depth and camera ego-motion from raw unlabeled RGB video streams is seeing exciting progress through self-supervision from strong geometric cues. To leverage not only appearance but also scene geometry, we propose a novel self-supervised two-stream network using RGB and inferred depth information for accurate visual odometry. In addition, we introduce a sparsity-inducing data augmentation policy for ego-motion learning that effectively regularizes the pose network to enable stronger generalization performance. As a result, we show that our proposed two-stream pose network achieves state-of-the-art results among learning-based methods on the KITTI odometry benchmark, and is especially suited for self-supervision at scale. Our experiments on a large-scale urban driving dataset of 1 million frames indicate that the performance of our proposed architecture does indeed scale progressively with more data.} }
Endnote
%0 Conference Paper %T Two Stream Networks for Self-Supervised Ego-Motion Estimation %A Rares Ambrus %A Vitor Guizilini %A Jie Li %A Sudeep Pillai Adrien Gaidon %B Proceedings of the Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2020 %E Leslie Pack Kaelbling %E Danica Kragic %E Komei Sugiura %F pmlr-v100-ambrus20a %I PMLR %J Proceedings of Machine Learning Research %P 1052--1061 %U http://proceedings.mlr.press %V 100 %W PMLR %X Learning depth and camera ego-motion from raw unlabeled RGB video streams is seeing exciting progress through self-supervision from strong geometric cues. To leverage not only appearance but also scene geometry, we propose a novel self-supervised two-stream network using RGB and inferred depth information for accurate visual odometry. In addition, we introduce a sparsity-inducing data augmentation policy for ego-motion learning that effectively regularizes the pose network to enable stronger generalization performance. As a result, we show that our proposed two-stream pose network achieves state-of-the-art results among learning-based methods on the KITTI odometry benchmark, and is especially suited for self-supervision at scale. Our experiments on a large-scale urban driving dataset of 1 million frames indicate that the performance of our proposed architecture does indeed scale progressively with more data.
APA
Ambrus, R., Guizilini, V., Li, J. & Gaidon, S.P.A.. (2020). Two Stream Networks for Self-Supervised Ego-Motion Estimation. Proceedings of the Conference on Robot Learning, in PMLR 100:1052-1061

Related Material