SURREAL: Open-Source Reinforcement Learning Framework and Robot Manipulation Benchmark

Linxi Fan, Yuke Zhu, Jiren Zhu, Zihua Liu, Orien Zeng, Anchit Gupta, Joan Creus-Costa, Silvio Savarese, Li Fei-Fei
Proceedings of The 2nd Conference on Robot Learning, PMLR 87:767-782, 2018.

Abstract

Reproducibility has been a significant challenge in deep reinforcement learning and robotics research. Open-source frameworks and standardized benchmarks can serve an integral role in rigorous evaluation and reproducible research. We introduce SURREAL, an open-source scalable framework that supports state-of-the-art distributed reinforcement learning algorithms. We design a principled distributed learning formulation that accommodates both on-policy and off-policy learning. We demonstrate that SURREAL algorithms outperform existing open-source implementations in both agent performance and learning efficiency. We also introduce SURREAL Robotics Suite, an accessible set of benchmarking tasks in physical simulation for reproducible robot manipulation research. We provide extensive evaluations of SURREAL algorithms and establish strong baseline results.

Cite this Paper

BibTeX
@InProceedings{pmlr-v87-fan18a, title = {SURREAL: Open-Source Reinforcement Learning Framework and Robot Manipulation Benchmark}, author = {Fan, Linxi and Zhu, Yuke and Zhu, Jiren and Liu, Zihua and Zeng, Orien and Gupta, Anchit and Creus-Costa, Joan and Savarese, Silvio and Fei-Fei, Li}, booktitle = {Proceedings of The 2nd Conference on Robot Learning}, pages = {767--782}, year = {2018}, editor = {Billard, Aude and Dragan, Anca and Peters, Jan and Morimoto, Jun}, volume = {87}, series = {Proceedings of Machine Learning Research}, month = {29--31 Oct}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v87/fan18a/fan18a.pdf}, url = {https://proceedings.mlr.press/v87/fan18a.html}, abstract = {Reproducibility has been a significant challenge in deep reinforcement learning and robotics research. Open-source frameworks and standardized benchmarks can serve an integral role in rigorous evaluation and reproducible research. We introduce SURREAL, an open-source scalable framework that supports state-of-the-art distributed reinforcement learning algorithms. We design a principled distributed learning formulation that accommodates both on-policy and off-policy learning. We demonstrate that SURREAL algorithms outperform existing open-source implementations in both agent performance and learning efficiency. We also introduce SURREAL Robotics Suite, an accessible set of benchmarking tasks in physical simulation for reproducible robot manipulation research. We provide extensive evaluations of SURREAL algorithms and establish strong baseline results.} }
Endnote
%0 Conference Paper %T SURREAL: Open-Source Reinforcement Learning Framework and Robot Manipulation Benchmark %A Linxi Fan %A Yuke Zhu %A Jiren Zhu %A Zihua Liu %A Orien Zeng %A Anchit Gupta %A Joan Creus-Costa %A Silvio Savarese %A Li Fei-Fei %B Proceedings of The 2nd Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2018 %E Aude Billard %E Anca Dragan %E Jan Peters %E Jun Morimoto %F pmlr-v87-fan18a %I PMLR %P 767--782 %U https://proceedings.mlr.press/v87/fan18a.html %V 87 %X Reproducibility has been a significant challenge in deep reinforcement learning and robotics research. Open-source frameworks and standardized benchmarks can serve an integral role in rigorous evaluation and reproducible research. We introduce SURREAL, an open-source scalable framework that supports state-of-the-art distributed reinforcement learning algorithms. We design a principled distributed learning formulation that accommodates both on-policy and off-policy learning. We demonstrate that SURREAL algorithms outperform existing open-source implementations in both agent performance and learning efficiency. We also introduce SURREAL Robotics Suite, an accessible set of benchmarking tasks in physical simulation for reproducible robot manipulation research. We provide extensive evaluations of SURREAL algorithms and establish strong baseline results.
APA
Fan, L., Zhu, Y., Zhu, J., Liu, Z., Zeng, O., Gupta, A., Creus-Costa, J., Savarese, S. & Fei-Fei, L.. (2018). SURREAL: Open-Source Reinforcement Learning Framework and Robot Manipulation Benchmark. Proceedings of The 2nd Conference on Robot Learning, in Proceedings of Machine Learning Research 87:767-782 Available from https://proceedings.mlr.press/v87/fan18a.html.

Related Material