Online Constrained Model-based Reinforcement Learning

Benjamin van Niekerk, Andreas Damianou, Benjamin Rosman
Proceedings of the 33rd Conference on Uncertainty in Artificial Intelligence, PMLR R15:611-620, 2017.

Abstract

Applying reinforcement learning to robotic systems poses a number of challenging prob- lems. A key requirement is the ability to han- dle continuous state and action spaces while remaining within a limited time and resource budget. Additionally, for safe operation, the system must make robust decisions under hard constraints. To address these challenges, we propose a model based approach that com- bines Gaussian Process regression and Reced- ing Horizon Control. Using sparse spectrum Gaussian Processes, we extend previous work by updating the dynamics model incrementally from a stream of sensory data. This results in an agent that can learn and plan in real-time under non-linear constraints. We test our ap- proach on a cart pole swing-up environment and demonstrate the benefits of online learn- ing on an autonomous racing task. The envi- ronment’s dynamics are learned from limited training data and can be reused in new task in- stances without retraining.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR15-niekerk17a, title = {Online Constrained Model-based Reinforcement Learning}, author = {van Niekerk, Benjamin and Damianou, Andreas and Rosman, Benjamin}, booktitle = {Proceedings of the 33rd Conference on Uncertainty in Artificial Intelligence}, pages = {611--620}, year = {2017}, editor = {Elidan, Gal and Kersting, Kristian}, volume = {R15}, series = {Proceedings of Machine Learning Research}, month = {11--15 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r15/main/assets/niekerk17a/niekerk17a.pdf}, url = {https://proceedings.mlr.press/r15/niekerk17a.html}, abstract = {Applying reinforcement learning to robotic systems poses a number of challenging prob- lems. A key requirement is the ability to han- dle continuous state and action spaces while remaining within a limited time and resource budget. Additionally, for safe operation, the system must make robust decisions under hard constraints. To address these challenges, we propose a model based approach that com- bines Gaussian Process regression and Reced- ing Horizon Control. Using sparse spectrum Gaussian Processes, we extend previous work by updating the dynamics model incrementally from a stream of sensory data. This results in an agent that can learn and plan in real-time under non-linear constraints. We test our ap- proach on a cart pole swing-up environment and demonstrate the benefits of online learn- ing on an autonomous racing task. The envi- ronment’s dynamics are learned from limited training data and can be reused in new task in- stances without retraining.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T Online Constrained Model-based Reinforcement Learning %A Benjamin van Niekerk %A Andreas Damianou %A Benjamin Rosman %B Proceedings of the 33rd Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2017 %E Gal Elidan %E Kristian Kersting %F pmlr-vR15-niekerk17a %I PMLR %P 611--620 %U https://proceedings.mlr.press/r15/niekerk17a.html %V R15 %X Applying reinforcement learning to robotic systems poses a number of challenging prob- lems. A key requirement is the ability to han- dle continuous state and action spaces while remaining within a limited time and resource budget. Additionally, for safe operation, the system must make robust decisions under hard constraints. To address these challenges, we propose a model based approach that com- bines Gaussian Process regression and Reced- ing Horizon Control. Using sparse spectrum Gaussian Processes, we extend previous work by updating the dynamics model incrementally from a stream of sensory data. This results in an agent that can learn and plan in real-time under non-linear constraints. We test our ap- proach on a cart pole swing-up environment and demonstrate the benefits of online learn- ing on an autonomous racing task. The envi- ronment’s dynamics are learned from limited training data and can be reused in new task in- stances without retraining. %Z Reissued by PMLR on 04 October 2026.
APA
van Niekerk, B., Damianou, A. & Rosman, B.. (2017). Online Constrained Model-based Reinforcement Learning. Proceedings of the 33rd Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R15:611-620 Available from https://proceedings.mlr.press/r15/niekerk17a.html. Reissued by PMLR on 04 October 2026.

Related Material