Context-Aware Deep Reinforcement Learning for Autonomous Robotic Navigation in Unknown Area

Jingsong Liang, Zhichen Wang, Yuhong Cao, Jimmy Chiun, Mengqi Zhang, Guillaume Adrien Sartoretti
Proceedings of The 7th Conference on Robot Learning, PMLR 229:1425-1436, 2023.

Abstract

Mapless navigation refers to a challenging task where a mobile robot must rapidly navigate to a predefined destination using its partial knowledge of the environment, which is updated online along the way, instead of a prior map of the environment. Inspired by the recent developments in deep reinforcement learning (DRL), we propose a learning-based framework for mapless navigation, which employs a context-aware policy network to achieve efficient decision-making (i.e., maximize the likelihood of finding the shortest route towards the target destination), especially in complex and large-scale environments. Specifically, our robot learns to form a context of its belief over the entire known area, which it uses to reason about long-term efficiency and sequence show-term movements. Additionally, we propose a graph rarefaction algorithm to enable more efficient decision-making in large-scale applications. We empirically demonstrate that our approach reduces average travel time by up to $61.4%$ and average planning time by up to $88.2%$ compared to benchmark planners (D*lite and BIT) on hundreds of test scenarios. We also validate our approach both in high-fidelity Gazebo simulations as well as on hardware, highlighting its promising applicability in the real world without further training/tuning.

Cite this Paper

BibTeX
@InProceedings{pmlr-v229-liang23a, title = {Context-Aware Deep Reinforcement Learning for Autonomous Robotic Navigation in Unknown Area}, author = {Liang, Jingsong and Wang, Zhichen and Cao, Yuhong and Chiun, Jimmy and Zhang, Mengqi and Sartoretti, Guillaume Adrien}, booktitle = {Proceedings of The 7th Conference on Robot Learning}, pages = {1425--1436}, year = {2023}, editor = {Tan, Jie and Toussaint, Marc and Darvish, Kourosh}, volume = {229}, series = {Proceedings of Machine Learning Research}, month = {06--09 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v229/liang23a/liang23a.pdf}, url = {https://proceedings.mlr.press/v229/liang23a.html}, abstract = {Mapless navigation refers to a challenging task where a mobile robot must rapidly navigate to a predefined destination using its partial knowledge of the environment, which is updated online along the way, instead of a prior map of the environment. Inspired by the recent developments in deep reinforcement learning (DRL), we propose a learning-based framework for mapless navigation, which employs a context-aware policy network to achieve efficient decision-making (i.e., maximize the likelihood of finding the shortest route towards the target destination), especially in complex and large-scale environments. Specifically, our robot learns to form a context of its belief over the entire known area, which it uses to reason about long-term efficiency and sequence show-term movements. Additionally, we propose a graph rarefaction algorithm to enable more efficient decision-making in large-scale applications. We empirically demonstrate that our approach reduces average travel time by up to $61.4%$ and average planning time by up to $88.2%$ compared to benchmark planners (D*lite and BIT) on hundreds of test scenarios. We also validate our approach both in high-fidelity Gazebo simulations as well as on hardware, highlighting its promising applicability in the real world without further training/tuning.} }
Endnote
%0 Conference Paper %T Context-Aware Deep Reinforcement Learning for Autonomous Robotic Navigation in Unknown Area %A Jingsong Liang %A Zhichen Wang %A Yuhong Cao %A Jimmy Chiun %A Mengqi Zhang %A Guillaume Adrien Sartoretti %B Proceedings of The 7th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2023 %E Jie Tan %E Marc Toussaint %E Kourosh Darvish %F pmlr-v229-liang23a %I PMLR %P 1425--1436 %U https://proceedings.mlr.press/v229/liang23a.html %V 229 %X Mapless navigation refers to a challenging task where a mobile robot must rapidly navigate to a predefined destination using its partial knowledge of the environment, which is updated online along the way, instead of a prior map of the environment. Inspired by the recent developments in deep reinforcement learning (DRL), we propose a learning-based framework for mapless navigation, which employs a context-aware policy network to achieve efficient decision-making (i.e., maximize the likelihood of finding the shortest route towards the target destination), especially in complex and large-scale environments. Specifically, our robot learns to form a context of its belief over the entire known area, which it uses to reason about long-term efficiency and sequence show-term movements. Additionally, we propose a graph rarefaction algorithm to enable more efficient decision-making in large-scale applications. We empirically demonstrate that our approach reduces average travel time by up to $61.4%$ and average planning time by up to $88.2%$ compared to benchmark planners (D*lite and BIT) on hundreds of test scenarios. We also validate our approach both in high-fidelity Gazebo simulations as well as on hardware, highlighting its promising applicability in the real world without further training/tuning.
APA
Liang, J., Wang, Z., Cao, Y., Chiun, J., Zhang, M. & Sartoretti, G.A.. (2023). Context-Aware Deep Reinforcement Learning for Autonomous Robotic Navigation in Unknown Area. Proceedings of The 7th Conference on Robot Learning, in Proceedings of Machine Learning Research 229:1425-1436 Available from https://proceedings.mlr.press/v229/liang23a.html.

Related Material