Anytime Depth Estimation with Limited Sensing and Computation Capabilities on Mobile Devices

Yuedong Yang, Zihui Xue, Radu Marculescu
Proceedings of the 5th Conference on Robot Learning, PMLR 164:609-618, 2022.

Abstract

Depth estimation is a safety critical and energy sensitive method for environment sensing. However, in real applications, the depth estimation may be halted at any time, due to the random interruptions or low energy capacity of battery when using powerful sensors like 3D LiDAR. To address this problem, we propose a depth estimation method that is robust to random halts and relies on energy-saving 2D LiDAR and a monocular camera. To this end, we formulate the depth estimation as an anytime problem and propose a new metric to evaluate its robustness under random interruptions. Our final model has only 2M parameters with a marginal accuracy loss compared to state-of-the-art baselines. Indeed, our experiments on NYU Depth v2 dataset show that our model is capable of processing 224$\times$224 resolution images and 2D point clouds with any computation budget larger than 6.37ms (157 FPS) and 0.2J on an NVIDIA Jetson TX2 system. Evaluations on KITTI dataset under supervised and self-supervised training show similar results.

Cite this Paper

BibTeX
@InProceedings{pmlr-v164-yang22b, title = {Anytime Depth Estimation with Limited Sensing and Computation Capabilities on Mobile Devices}, author = {Yang, Yuedong and Xue, Zihui and Marculescu, Radu}, booktitle = {Proceedings of the 5th Conference on Robot Learning}, pages = {609--618}, year = {2022}, editor = {Faust, Aleksandra and Hsu, David and Neumann, Gerhard}, volume = {164}, series = {Proceedings of Machine Learning Research}, month = {08--11 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v164/yang22b/yang22b.pdf}, url = {https://proceedings.mlr.press/v164/yang22b.html}, abstract = {Depth estimation is a safety critical and energy sensitive method for environment sensing. However, in real applications, the depth estimation may be halted at any time, due to the random interruptions or low energy capacity of battery when using powerful sensors like 3D LiDAR. To address this problem, we propose a depth estimation method that is robust to random halts and relies on energy-saving 2D LiDAR and a monocular camera. To this end, we formulate the depth estimation as an anytime problem and propose a new metric to evaluate its robustness under random interruptions. Our final model has only 2M parameters with a marginal accuracy loss compared to state-of-the-art baselines. Indeed, our experiments on NYU Depth v2 dataset show that our model is capable of processing 224$\times$224 resolution images and 2D point clouds with any computation budget larger than 6.37ms (157 FPS) and 0.2J on an NVIDIA Jetson TX2 system. Evaluations on KITTI dataset under supervised and self-supervised training show similar results.} }
Endnote
%0 Conference Paper %T Anytime Depth Estimation with Limited Sensing and Computation Capabilities on Mobile Devices %A Yuedong Yang %A Zihui Xue %A Radu Marculescu %B Proceedings of the 5th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2022 %E Aleksandra Faust %E David Hsu %E Gerhard Neumann %F pmlr-v164-yang22b %I PMLR %P 609--618 %U https://proceedings.mlr.press/v164/yang22b.html %V 164 %X Depth estimation is a safety critical and energy sensitive method for environment sensing. However, in real applications, the depth estimation may be halted at any time, due to the random interruptions or low energy capacity of battery when using powerful sensors like 3D LiDAR. To address this problem, we propose a depth estimation method that is robust to random halts and relies on energy-saving 2D LiDAR and a monocular camera. To this end, we formulate the depth estimation as an anytime problem and propose a new metric to evaluate its robustness under random interruptions. Our final model has only 2M parameters with a marginal accuracy loss compared to state-of-the-art baselines. Indeed, our experiments on NYU Depth v2 dataset show that our model is capable of processing 224$\times$224 resolution images and 2D point clouds with any computation budget larger than 6.37ms (157 FPS) and 0.2J on an NVIDIA Jetson TX2 system. Evaluations on KITTI dataset under supervised and self-supervised training show similar results.
APA
Yang, Y., Xue, Z. & Marculescu, R.. (2022). Anytime Depth Estimation with Limited Sensing and Computation Capabilities on Mobile Devices. Proceedings of the 5th Conference on Robot Learning, in Proceedings of Machine Learning Research 164:609-618 Available from https://proceedings.mlr.press/v164/yang22b.html.

Related Material