Hierarchical Reinforcement Learning with Parameters

Maciej Klimek, Henryk Michalewski, Piotr Mi\loś
Proceedings of the 1st Annual Conference on Robot Learning, PMLR 78:301-313, 2017.

Abstract

In this work we introduce and evaluate a model of Hierarchical Reinforcement Learning with Parameters. In the first stage we train agents to execute relatively simple actions like reaching or gripping. In the second stage we train a hierarchical manager to compose these actions to solve more complicated tasks. The manager may pass parameters to agents thus controlling details of undertaken actions. The hierarchical approach with parameters can be used with any optimization algorithm. In this work we adapt to our setting methods described in [1]. We show that their theoretical foundation, including monotonicity of improvements, still holds. We experimentally compare the hierarchical reinforcement learning with the standard, non-hierarchical approach and conclude that the hierarchical learning with parameters is a viable way to improve final results and stability of learning.

Cite this Paper

BibTeX
@InProceedings{pmlr-v78-klimek17a, title = {Hierarchical Reinforcement Learning with Parameters}, author = {Klimek, Maciej and Michalewski, Henryk and Mi\loś, Piotr}, booktitle = {Proceedings of the 1st Annual Conference on Robot Learning}, pages = {301--313}, year = {2017}, editor = {Levine, Sergey and Vanhoucke, Vincent and Goldberg, Ken}, volume = {78}, series = {Proceedings of Machine Learning Research}, month = {13--15 Nov}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v78/klimek17a/klimek17a.pdf}, url = {https://proceedings.mlr.press/v78/klimek17a.html}, abstract = {In this work we introduce and evaluate a model of Hierarchical Reinforcement Learning with Parameters. In the first stage we train agents to execute relatively simple actions like reaching or gripping. In the second stage we train a hierarchical manager to compose these actions to solve more complicated tasks. The manager may pass parameters to agents thus controlling details of undertaken actions. The hierarchical approach with parameters can be used with any optimization algorithm. In this work we adapt to our setting methods described in [1]. We show that their theoretical foundation, including monotonicity of improvements, still holds. We experimentally compare the hierarchical reinforcement learning with the standard, non-hierarchical approach and conclude that the hierarchical learning with parameters is a viable way to improve final results and stability of learning.} }
Endnote
%0 Conference Paper %T Hierarchical Reinforcement Learning with Parameters %A Maciej Klimek %A Henryk Michalewski %A Piotr Mi\loś %B Proceedings of the 1st Annual Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2017 %E Sergey Levine %E Vincent Vanhoucke %E Ken Goldberg %F pmlr-v78-klimek17a %I PMLR %P 301--313 %U https://proceedings.mlr.press/v78/klimek17a.html %V 78 %X In this work we introduce and evaluate a model of Hierarchical Reinforcement Learning with Parameters. In the first stage we train agents to execute relatively simple actions like reaching or gripping. In the second stage we train a hierarchical manager to compose these actions to solve more complicated tasks. The manager may pass parameters to agents thus controlling details of undertaken actions. The hierarchical approach with parameters can be used with any optimization algorithm. In this work we adapt to our setting methods described in [1]. We show that their theoretical foundation, including monotonicity of improvements, still holds. We experimentally compare the hierarchical reinforcement learning with the standard, non-hierarchical approach and conclude that the hierarchical learning with parameters is a viable way to improve final results and stability of learning.
APA
Klimek, M., Michalewski, H. & Mi\loś, P.. (2017). Hierarchical Reinforcement Learning with Parameters. Proceedings of the 1st Annual Conference on Robot Learning, in Proceedings of Machine Learning Research 78:301-313 Available from https://proceedings.mlr.press/v78/klimek17a.html.

Related Material