Learning Coverage Paths in Unknown Environments with Deep Reinforcement Learning

Arvi Jonnarth, Jie Zhao, Michael Felsberg
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:22491-22508, 2024.

Abstract

Coverage path planning (CPP) is the problem of finding a path that covers the entire free space of a confined area, with applications ranging from robotic lawn mowing to search-and-rescue. When the environment is unknown, the path needs to be planned online while mapping the environment, which cannot be addressed by offline planning methods that do not allow for a flexible path space. We investigate how suitable reinforcement learning is for this challenging problem, and analyze the involved components required to efficiently learn coverage paths, such as action space, input feature representation, neural network architecture, and reward function. We propose a computationally feasible egocentric map representation based on frontiers, and a novel reward term based on total variation to promote complete coverage. Through extensive experiments, we show that our approach surpasses the performance of both previous RL-based approaches and highly specialized methods across multiple CPP variations.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-jonnarth24a, title = {Learning Coverage Paths in Unknown Environments with Deep Reinforcement Learning}, author = {Jonnarth, Arvi and Zhao, Jie and Felsberg, Michael}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {22491--22508}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/jonnarth24a/jonnarth24a.pdf}, url = {https://proceedings.mlr.press/v235/jonnarth24a.html}, abstract = {Coverage path planning (CPP) is the problem of finding a path that covers the entire free space of a confined area, with applications ranging from robotic lawn mowing to search-and-rescue. When the environment is unknown, the path needs to be planned online while mapping the environment, which cannot be addressed by offline planning methods that do not allow for a flexible path space. We investigate how suitable reinforcement learning is for this challenging problem, and analyze the involved components required to efficiently learn coverage paths, such as action space, input feature representation, neural network architecture, and reward function. We propose a computationally feasible egocentric map representation based on frontiers, and a novel reward term based on total variation to promote complete coverage. Through extensive experiments, we show that our approach surpasses the performance of both previous RL-based approaches and highly specialized methods across multiple CPP variations.} }
Endnote
%0 Conference Paper %T Learning Coverage Paths in Unknown Environments with Deep Reinforcement Learning %A Arvi Jonnarth %A Jie Zhao %A Michael Felsberg %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-jonnarth24a %I PMLR %P 22491--22508 %U https://proceedings.mlr.press/v235/jonnarth24a.html %V 235 %X Coverage path planning (CPP) is the problem of finding a path that covers the entire free space of a confined area, with applications ranging from robotic lawn mowing to search-and-rescue. When the environment is unknown, the path needs to be planned online while mapping the environment, which cannot be addressed by offline planning methods that do not allow for a flexible path space. We investigate how suitable reinforcement learning is for this challenging problem, and analyze the involved components required to efficiently learn coverage paths, such as action space, input feature representation, neural network architecture, and reward function. We propose a computationally feasible egocentric map representation based on frontiers, and a novel reward term based on total variation to promote complete coverage. Through extensive experiments, we show that our approach surpasses the performance of both previous RL-based approaches and highly specialized methods across multiple CPP variations.
APA
Jonnarth, A., Zhao, J. & Felsberg, M.. (2024). Learning Coverage Paths in Unknown Environments with Deep Reinforcement Learning. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:22491-22508 Available from https://proceedings.mlr.press/v235/jonnarth24a.html.

Related Material