Hierarchically Integrated Models: Learning to Navigate from Heterogeneous Robots

Katie Kang, Gregory Kahn, Sergey Levine
Proceedings of the 5th Conference on Robot Learning, PMLR 164:1316-1325, 2022.

Abstract

Deep reinforcement learning algorithms require large and diverse datasets in order to learn successful policies for perception-based mobile navigation. However, gathering such datasets with a single robot can be prohibitively expensive. Collecting data with multiple different robotic platforms with possibly different dynamics is a more scalable approach to large-scale data collection. But how can deep reinforcement learning algorithms leverage such heterogeneous datasets? In this work, we propose a deep reinforcement learning algorithm with hierarchically integrated models (HInt). At training time, HInt learns separate perception and dynamics models, and at test time, HInt integrates the two models in a hierarchical manner and plans actions with the integrated model. This method of planning with hierarchically integrated models allows the algorithm to train on datasets gathered by a variety of different platforms, while respecting the physical capabilities of the deployment robot at test time. Our mobile navigation experiments show that HInt outperforms conventional hierarchical policies and single-source approaches.

Cite this Paper

BibTeX
@InProceedings{pmlr-v164-kang22a, title = {Hierarchically Integrated Models: Learning to Navigate from Heterogeneous Robots}, author = {Kang, Katie and Kahn, Gregory and Levine, Sergey}, booktitle = {Proceedings of the 5th Conference on Robot Learning}, pages = {1316--1325}, year = {2022}, editor = {Faust, Aleksandra and Hsu, David and Neumann, Gerhard}, volume = {164}, series = {Proceedings of Machine Learning Research}, month = {08--11 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v164/kang22a/kang22a.pdf}, url = {https://proceedings.mlr.press/v164/kang22a.html}, abstract = {Deep reinforcement learning algorithms require large and diverse datasets in order to learn successful policies for perception-based mobile navigation. However, gathering such datasets with a single robot can be prohibitively expensive. Collecting data with multiple different robotic platforms with possibly different dynamics is a more scalable approach to large-scale data collection. But how can deep reinforcement learning algorithms leverage such heterogeneous datasets? In this work, we propose a deep reinforcement learning algorithm with hierarchically integrated models (HInt). At training time, HInt learns separate perception and dynamics models, and at test time, HInt integrates the two models in a hierarchical manner and plans actions with the integrated model. This method of planning with hierarchically integrated models allows the algorithm to train on datasets gathered by a variety of different platforms, while respecting the physical capabilities of the deployment robot at test time. Our mobile navigation experiments show that HInt outperforms conventional hierarchical policies and single-source approaches.} }
Endnote
%0 Conference Paper %T Hierarchically Integrated Models: Learning to Navigate from Heterogeneous Robots %A Katie Kang %A Gregory Kahn %A Sergey Levine %B Proceedings of the 5th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2022 %E Aleksandra Faust %E David Hsu %E Gerhard Neumann %F pmlr-v164-kang22a %I PMLR %P 1316--1325 %U https://proceedings.mlr.press/v164/kang22a.html %V 164 %X Deep reinforcement learning algorithms require large and diverse datasets in order to learn successful policies for perception-based mobile navigation. However, gathering such datasets with a single robot can be prohibitively expensive. Collecting data with multiple different robotic platforms with possibly different dynamics is a more scalable approach to large-scale data collection. But how can deep reinforcement learning algorithms leverage such heterogeneous datasets? In this work, we propose a deep reinforcement learning algorithm with hierarchically integrated models (HInt). At training time, HInt learns separate perception and dynamics models, and at test time, HInt integrates the two models in a hierarchical manner and plans actions with the integrated model. This method of planning with hierarchically integrated models allows the algorithm to train on datasets gathered by a variety of different platforms, while respecting the physical capabilities of the deployment robot at test time. Our mobile navigation experiments show that HInt outperforms conventional hierarchical policies and single-source approaches.
APA
Kang, K., Kahn, G. & Levine, S.. (2022). Hierarchically Integrated Models: Learning to Navigate from Heterogeneous Robots. Proceedings of the 5th Conference on Robot Learning, in Proceedings of Machine Learning Research 164:1316-1325 Available from https://proceedings.mlr.press/v164/kang22a.html.

Related Material