CORL: a continuous-state offset-dynamics reinforcement learner

Emma Brunskill, Bethany R. Leffler, Lihong Li, Michael L. Littman, Nicholas Roy
Proceedings of the 24th Conference on Uncertainty in Artificial Intelligence, PMLR R6:53-61, 2008.

Abstract

Continuous state spaces and stochastic, switching dynamics characterize a number of rich, real-world domains, such as robot navigation across varying terrain. We describe a reinforcement-learning algorithm for learning in these domains and prove for certain environments the algorithm is probably approximately correct with a sample complexity that scales polynomially with the state-space dimension. Unfortunately, no optimal planning techniques exist in general for such problems; instead we use fitted value iteration to solve the learned MDP, and include the error due to approximate planning in our bounds. Finally, we report an experiment using a robotic car driving over varying terrain to demonstrate that these dynamics representations adequately capture real-world dynamics and that our algorithm can be used to efficiently solve such problems.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR6-brunskill08a, title = {CORL: a continuous-state offset-dynamics reinforcement learner}, author = {Brunskill, Emma and Leffler, Bethany R. and Li, Lihong and Littman, Michael L. and Roy, Nicholas}, booktitle = {Proceedings of the 24th Conference on Uncertainty in Artificial Intelligence}, pages = {53--61}, year = {2008}, editor = {McAllester, David A. and Myllymäki, Petri}, volume = {R6}, series = {Proceedings of Machine Learning Research}, month = {09--12 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r6/main/assets/brunskill08a/brunskill08a.pdf}, url = {https://proceedings.mlr.press/r6/brunskill08a.html}, abstract = {Continuous state spaces and stochastic, switching dynamics characterize a number of rich, real-world domains, such as robot navigation across varying terrain. We describe a reinforcement-learning algorithm for learning in these domains and prove for certain environments the algorithm is probably approximately correct with a sample complexity that scales polynomially with the state-space dimension. Unfortunately, no optimal planning techniques exist in general for such problems; instead we use fitted value iteration to solve the learned MDP, and include the error due to approximate planning in our bounds. Finally, we report an experiment using a robotic car driving over varying terrain to demonstrate that these dynamics representations adequately capture real-world dynamics and that our algorithm can be used to efficiently solve such problems.}, note = {Reissued by PMLR on 09 October 2024.} }
Endnote
%0 Conference Paper %T CORL: a continuous-state offset-dynamics reinforcement learner %A Emma Brunskill %A Bethany R. Leffler %A Lihong Li %A Michael L. Littman %A Nicholas Roy %B Proceedings of the 24th Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2008 %E David A. McAllester %E Petri Myllymäki %F pmlr-vR6-brunskill08a %I PMLR %P 53--61 %U https://proceedings.mlr.press/r6/brunskill08a.html %V R6 %X Continuous state spaces and stochastic, switching dynamics characterize a number of rich, real-world domains, such as robot navigation across varying terrain. We describe a reinforcement-learning algorithm for learning in these domains and prove for certain environments the algorithm is probably approximately correct with a sample complexity that scales polynomially with the state-space dimension. Unfortunately, no optimal planning techniques exist in general for such problems; instead we use fitted value iteration to solve the learned MDP, and include the error due to approximate planning in our bounds. Finally, we report an experiment using a robotic car driving over varying terrain to demonstrate that these dynamics representations adequately capture real-world dynamics and that our algorithm can be used to efficiently solve such problems. %Z Reissued by PMLR on 09 October 2024.
APA
Brunskill, E., Leffler, B.R., Li, L., Littman, M.L. & Roy, N.. (2008). CORL: a continuous-state offset-dynamics reinforcement learner. Proceedings of the 24th Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R6:53-61 Available from https://proceedings.mlr.press/r6/brunskill08a.html. Reissued by PMLR on 09 October 2024.

Related Material