Benchmarking Real-Time Reinforcement Learning

Pierre Thodoroff, Wenyu Li, Neil D. Lawrence
NeurIPS 2021 Workshop on Pre-registration in Machine Learning, PMLR 181:26-41, 2022.

Abstract

Decision-making algorithms can require fast response time in applications as diverse as self-driving cars and minimizing load times of webpages. Yet, modern algorithms (deep reinforcement learning) are usually developed in scenarios where inference and training computational costs are ignored. This proposal aims to study reinforcement learning and control algorithms for real-time continuous control. In this scenario, the environment continuously evolves while actions are being computed by the agent (either in training or inference). The first goal is to provide a clear picture of the performance of modern algorithms modulated by their computational costs. The second goal is to identify the major challenges that arise when considering real-time environments to guide further research.

Cite this Paper

BibTeX
@InProceedings{pmlr-v181-thodoroff22a, title = {Benchmarking Real-Time Reinforcement Learning}, author = {Thodoroff, Pierre and Li, Wenyu and Lawrence, Neil D.}, booktitle = {NeurIPS 2021 Workshop on Pre-registration in Machine Learning}, pages = {26--41}, year = {2022}, editor = {Albanie, Samuel and Henriques, João F. and Bertinetto, Luca and Hernández-Garcı́a, Alex and Doughty, Hazel and Varol, Gül}, volume = {181}, series = {Proceedings of Machine Learning Research}, month = {13 Dec}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v181/thodoroff22a/thodoroff22a.pdf}, url = {https://proceedings.mlr.press/v181/thodoroff22a.html}, abstract = {Decision-making algorithms can require fast response time in applications as diverse as self-driving cars and minimizing load times of webpages. Yet, modern algorithms (deep reinforcement learning) are usually developed in scenarios where inference and training computational costs are ignored. This proposal aims to study reinforcement learning and control algorithms for real-time continuous control. In this scenario, the environment continuously evolves while actions are being computed by the agent (either in training or inference). The first goal is to provide a clear picture of the performance of modern algorithms modulated by their computational costs. The second goal is to identify the major challenges that arise when considering real-time environments to guide further research.} }
Endnote
%0 Conference Paper %T Benchmarking Real-Time Reinforcement Learning %A Pierre Thodoroff %A Wenyu Li %A Neil D. Lawrence %B NeurIPS 2021 Workshop on Pre-registration in Machine Learning %C Proceedings of Machine Learning Research %D 2022 %E Samuel Albanie %E João F. Henriques %E Luca Bertinetto %E Alex Hernández-Garcı́a %E Hazel Doughty %E Gül Varol %F pmlr-v181-thodoroff22a %I PMLR %P 26--41 %U https://proceedings.mlr.press/v181/thodoroff22a.html %V 181 %X Decision-making algorithms can require fast response time in applications as diverse as self-driving cars and minimizing load times of webpages. Yet, modern algorithms (deep reinforcement learning) are usually developed in scenarios where inference and training computational costs are ignored. This proposal aims to study reinforcement learning and control algorithms for real-time continuous control. In this scenario, the environment continuously evolves while actions are being computed by the agent (either in training or inference). The first goal is to provide a clear picture of the performance of modern algorithms modulated by their computational costs. The second goal is to identify the major challenges that arise when considering real-time environments to guide further research.
APA
Thodoroff, P., Li, W. & Lawrence, N.D.. (2022). Benchmarking Real-Time Reinforcement Learning. NeurIPS 2021 Workshop on Pre-registration in Machine Learning, in Proceedings of Machine Learning Research 181:26-41 Available from https://proceedings.mlr.press/v181/thodoroff22a.html.

Related Material