Deep Reactive Planning in Dynamic Environments

Kei Ota, Devesh Jha, Tadashi Onishi, Asako Kanezaki, Yusuke Yoshiyasu, Yoko Sasaki, Toshisada Mariyama, Daniel Nikovski
Proceedings of the 2020 Conference on Robot Learning, PMLR 155:1943-1957, 2021.

Abstract

The main novelty of the proposed approach is that it allows a robot to learn an end-to-end policy which can adapt to changes in the environment during execution. While goal conditioning of policies has been studied in the RL literature, such approaches are not easily extended to settings where the robot’s goal can change during execution. This is something that humans are naturally able to do. However, it is difficult for robots to learn such reflexes (i.e., to naturally respond to dynamic environments), especially when the goal location is not explicitly provided to the robot, and instead needs to be perceived through a vision sensor. In the current work, we present a method that can achieve such behavior by combining traditional kinematic planning, deep learning, and deep reinforcement learning in a synergistic fashion to generalize to arbitrary environments. We demonstrate the proposed approach for several reaching and pick-and-place tasks in simulation, as well as on a real system of a 6-DoF industrial manipulator.

Cite this Paper

BibTeX
@InProceedings{pmlr-v155-ota21a, title = {Deep Reactive Planning in Dynamic Environments}, author = {Ota, Kei and Jha, Devesh and Onishi, Tadashi and Kanezaki, Asako and Yoshiyasu, Yusuke and Sasaki, Yoko and Mariyama, Toshisada and Nikovski, Daniel}, booktitle = {Proceedings of the 2020 Conference on Robot Learning}, pages = {1943--1957}, year = {2021}, editor = {Kober, Jens and Ramos, Fabio and Tomlin, Claire}, volume = {155}, series = {Proceedings of Machine Learning Research}, month = {16--18 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v155/ota21a/ota21a.pdf}, url = {https://proceedings.mlr.press/v155/ota21a.html}, abstract = {The main novelty of the proposed approach is that it allows a robot to learn an end-to-end policy which can adapt to changes in the environment during execution. While goal conditioning of policies has been studied in the RL literature, such approaches are not easily extended to settings where the robot’s goal can change during execution. This is something that humans are naturally able to do. However, it is difficult for robots to learn such reflexes (i.e., to naturally respond to dynamic environments), especially when the goal location is not explicitly provided to the robot, and instead needs to be perceived through a vision sensor. In the current work, we present a method that can achieve such behavior by combining traditional kinematic planning, deep learning, and deep reinforcement learning in a synergistic fashion to generalize to arbitrary environments. We demonstrate the proposed approach for several reaching and pick-and-place tasks in simulation, as well as on a real system of a 6-DoF industrial manipulator.} }
Endnote
%0 Conference Paper %T Deep Reactive Planning in Dynamic Environments %A Kei Ota %A Devesh Jha %A Tadashi Onishi %A Asako Kanezaki %A Yusuke Yoshiyasu %A Yoko Sasaki %A Toshisada Mariyama %A Daniel Nikovski %B Proceedings of the 2020 Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2021 %E Jens Kober %E Fabio Ramos %E Claire Tomlin %F pmlr-v155-ota21a %I PMLR %P 1943--1957 %U https://proceedings.mlr.press/v155/ota21a.html %V 155 %X The main novelty of the proposed approach is that it allows a robot to learn an end-to-end policy which can adapt to changes in the environment during execution. While goal conditioning of policies has been studied in the RL literature, such approaches are not easily extended to settings where the robot’s goal can change during execution. This is something that humans are naturally able to do. However, it is difficult for robots to learn such reflexes (i.e., to naturally respond to dynamic environments), especially when the goal location is not explicitly provided to the robot, and instead needs to be perceived through a vision sensor. In the current work, we present a method that can achieve such behavior by combining traditional kinematic planning, deep learning, and deep reinforcement learning in a synergistic fashion to generalize to arbitrary environments. We demonstrate the proposed approach for several reaching and pick-and-place tasks in simulation, as well as on a real system of a 6-DoF industrial manipulator.
APA
Ota, K., Jha, D., Onishi, T., Kanezaki, A., Yoshiyasu, Y., Sasaki, Y., Mariyama, T. & Nikovski, D.. (2021). Deep Reactive Planning in Dynamic Environments. Proceedings of the 2020 Conference on Robot Learning, in Proceedings of Machine Learning Research 155:1943-1957 Available from https://proceedings.mlr.press/v155/ota21a.html.

Related Material