Policy learning in SE(3) action spaces

Dian Wang, Colin Kohler, Robert Platt
Proceedings of the 2020 Conference on Robot Learning, PMLR 155:1481-1497, 2021.

Abstract

In the spatial action representation, the action space spans the space of target poses for robot motion commands, i.e. SE(2) or SE(3). This approach has been used to solve challenging robotic manipulation problems and shows promise. However, the method is often limited to a three dimensional action space and short horizon tasks. This paper proposes ASRSE3, a new method for handling higher dimensional spatial action spaces that transforms an original MDP with high dimensional action space into a new MDP with reduced action space and augmented state space. We also propose SDQfD, a variation of DQfD designed for large action spaces. ASRSE3 and SDQfD are evaluated in the context of a set of challenging block construction tasks. We show that both methods outperform standard baselines and can be used in practice on real robotics systems.

Cite this Paper

BibTeX
@InProceedings{pmlr-v155-wang21f, title = {Policy learning in SE(3) action spaces}, author = {Wang, Dian and Kohler, Colin and Platt, Robert}, booktitle = {Proceedings of the 2020 Conference on Robot Learning}, pages = {1481--1497}, year = {2021}, editor = {Kober, Jens and Ramos, Fabio and Tomlin, Claire}, volume = {155}, series = {Proceedings of Machine Learning Research}, month = {16--18 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v155/wang21f/wang21f.pdf}, url = {https://proceedings.mlr.press/v155/wang21f.html}, abstract = {In the spatial action representation, the action space spans the space of target poses for robot motion commands, i.e. SE(2) or SE(3). This approach has been used to solve challenging robotic manipulation problems and shows promise. However, the method is often limited to a three dimensional action space and short horizon tasks. This paper proposes ASRSE3, a new method for handling higher dimensional spatial action spaces that transforms an original MDP with high dimensional action space into a new MDP with reduced action space and augmented state space. We also propose SDQfD, a variation of DQfD designed for large action spaces. ASRSE3 and SDQfD are evaluated in the context of a set of challenging block construction tasks. We show that both methods outperform standard baselines and can be used in practice on real robotics systems.} }
Endnote
%0 Conference Paper %T Policy learning in SE(3) action spaces %A Dian Wang %A Colin Kohler %A Robert Platt %B Proceedings of the 2020 Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2021 %E Jens Kober %E Fabio Ramos %E Claire Tomlin %F pmlr-v155-wang21f %I PMLR %P 1481--1497 %U https://proceedings.mlr.press/v155/wang21f.html %V 155 %X In the spatial action representation, the action space spans the space of target poses for robot motion commands, i.e. SE(2) or SE(3). This approach has been used to solve challenging robotic manipulation problems and shows promise. However, the method is often limited to a three dimensional action space and short horizon tasks. This paper proposes ASRSE3, a new method for handling higher dimensional spatial action spaces that transforms an original MDP with high dimensional action space into a new MDP with reduced action space and augmented state space. We also propose SDQfD, a variation of DQfD designed for large action spaces. ASRSE3 and SDQfD are evaluated in the context of a set of challenging block construction tasks. We show that both methods outperform standard baselines and can be used in practice on real robotics systems.
APA
Wang, D., Kohler, C. & Platt, R.. (2021). Policy learning in SE(3) action spaces. Proceedings of the 2020 Conference on Robot Learning, in Proceedings of Machine Learning Research 155:1481-1497 Available from https://proceedings.mlr.press/v155/wang21f.html.

Related Material