A Survey on Reproducibility by Evaluating Deep Reinforcement Learning Algorithms on Real-World Robots

Nicolai A. Lynnerup, Laura Nolling, Rasmus Hasle, John Hallam
Proceedings of the Conference on Robot Learning, PMLR 100:466-489, 2020.

Abstract

As reinforcement learning (RL) achieves more success in solving complex tasks, more care is needed to ensure that RL research is reproducible and that algorithms therein can be compared easily and fairly with minimal bias. RL results are, however, notoriously hard to reproduce due to the algorithms’ intrinsic variance, the environments’ stochasticity, and numerous (potentially unreported) hyper-parameters. In this work we investigate the many issues leading to irreproducible research and how to manage those. We further show how to utilise a rigorous and standardised evaluation approach for easing the process of documentation, evaluation and fair comparison of different algorithms, where we emphasise the importance of choosing the right measurement metrics and conducting proper statistics on the results, for unbiased reporting of the results.

Cite this Paper

BibTeX
@InProceedings{pmlr-v100-lynnerup20a, title = {A Survey on Reproducibility by Evaluating Deep Reinforcement Learning Algorithms on Real-World Robots}, author = {Lynnerup, Nicolai A. and Nolling, Laura and Hasle, Rasmus and Hallam, John}, booktitle = {Proceedings of the Conference on Robot Learning}, pages = {466--489}, year = {2020}, editor = {Kaelbling, Leslie Pack and Kragic, Danica and Sugiura, Komei}, volume = {100}, series = {Proceedings of Machine Learning Research}, month = {30 Oct--01 Nov}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v100/lynnerup20a/lynnerup20a.pdf}, url = {https://proceedings.mlr.press/v100/lynnerup20a.html}, abstract = {As reinforcement learning (RL) achieves more success in solving complex tasks, more care is needed to ensure that RL research is reproducible and that algorithms therein can be compared easily and fairly with minimal bias. RL results are, however, notoriously hard to reproduce due to the algorithms’ intrinsic variance, the environments’ stochasticity, and numerous (potentially unreported) hyper-parameters. In this work we investigate the many issues leading to irreproducible research and how to manage those. We further show how to utilise a rigorous and standardised evaluation approach for easing the process of documentation, evaluation and fair comparison of different algorithms, where we emphasise the importance of choosing the right measurement metrics and conducting proper statistics on the results, for unbiased reporting of the results.} }
Endnote
%0 Conference Paper %T A Survey on Reproducibility by Evaluating Deep Reinforcement Learning Algorithms on Real-World Robots %A Nicolai A. Lynnerup %A Laura Nolling %A Rasmus Hasle %A John Hallam %B Proceedings of the Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2020 %E Leslie Pack Kaelbling %E Danica Kragic %E Komei Sugiura %F pmlr-v100-lynnerup20a %I PMLR %P 466--489 %U https://proceedings.mlr.press/v100/lynnerup20a.html %V 100 %X As reinforcement learning (RL) achieves more success in solving complex tasks, more care is needed to ensure that RL research is reproducible and that algorithms therein can be compared easily and fairly with minimal bias. RL results are, however, notoriously hard to reproduce due to the algorithms’ intrinsic variance, the environments’ stochasticity, and numerous (potentially unreported) hyper-parameters. In this work we investigate the many issues leading to irreproducible research and how to manage those. We further show how to utilise a rigorous and standardised evaluation approach for easing the process of documentation, evaluation and fair comparison of different algorithms, where we emphasise the importance of choosing the right measurement metrics and conducting proper statistics on the results, for unbiased reporting of the results.
APA
Lynnerup, N.A., Nolling, L., Hasle, R. & Hallam, J.. (2020). A Survey on Reproducibility by Evaluating Deep Reinforcement Learning Algorithms on Real-World Robots. Proceedings of the Conference on Robot Learning, in Proceedings of Machine Learning Research 100:466-489 Available from https://proceedings.mlr.press/v100/lynnerup20a.html.

Related Material