Adaptable replanning with compressed linear action models for learning from demonstrations

Clement Gehring, Leslie Pack Kaelbling, Tomas Lozano-Perez
Proceedings of The 2nd Conference on Robot Learning, PMLR 87:432-442, 2018.

Abstract

We propose an adaptable and efficient model-based reinforcement learning approach well suited for continuous domains with sparse samples, a setting often encountered when learning from demonstrations. The flexibility of our method originates from the approximate transition models, estimated from data, and the online replanning approach proposed. Together, these components allow for immediate adaptation to a new task, given in the form of a reward function. The efficiency of our method comes from two approximations. First, rather than representing a complete distribution over the results of taking an action, which is difficult in continuous state spaces, it learns a linear model of the expected transition for each action. Second, it uses a novel strategy for compressing these linear action models, which significantly reduces space and time for learning models, and supports efficient online generation of open-loop plans. The effectiveness of these methods is demonstrated in a simulated driving domain with a 20-dimensional continuous input space.

Cite this Paper

BibTeX
@InProceedings{pmlr-v87-gehring18a, title = {Adaptable replanning with compressed linear action models for learning from demonstrations}, author = {Gehring, Clement and Kaelbling, Leslie Pack and Lozano-Perez, Tomas}, booktitle = {Proceedings of The 2nd Conference on Robot Learning}, pages = {432--442}, year = {2018}, editor = {Billard, Aude and Dragan, Anca and Peters, Jan and Morimoto, Jun}, volume = {87}, series = {Proceedings of Machine Learning Research}, month = {29--31 Oct}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v87/gehring18a/gehring18a.pdf}, url = {https://proceedings.mlr.press/v87/gehring18a.html}, abstract = {We propose an adaptable and efficient model-based reinforcement learning approach well suited for continuous domains with sparse samples, a setting often encountered when learning from demonstrations. The flexibility of our method originates from the approximate transition models, estimated from data, and the online replanning approach proposed. Together, these components allow for immediate adaptation to a new task, given in the form of a reward function. The efficiency of our method comes from two approximations. First, rather than representing a complete distribution over the results of taking an action, which is difficult in continuous state spaces, it learns a linear model of the expected transition for each action. Second, it uses a novel strategy for compressing these linear action models, which significantly reduces space and time for learning models, and supports efficient online generation of open-loop plans. The effectiveness of these methods is demonstrated in a simulated driving domain with a 20-dimensional continuous input space. } }
Endnote
%0 Conference Paper %T Adaptable replanning with compressed linear action models for learning from demonstrations %A Clement Gehring %A Leslie Pack Kaelbling %A Tomas Lozano-Perez %B Proceedings of The 2nd Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2018 %E Aude Billard %E Anca Dragan %E Jan Peters %E Jun Morimoto %F pmlr-v87-gehring18a %I PMLR %P 432--442 %U https://proceedings.mlr.press/v87/gehring18a.html %V 87 %X We propose an adaptable and efficient model-based reinforcement learning approach well suited for continuous domains with sparse samples, a setting often encountered when learning from demonstrations. The flexibility of our method originates from the approximate transition models, estimated from data, and the online replanning approach proposed. Together, these components allow for immediate adaptation to a new task, given in the form of a reward function. The efficiency of our method comes from two approximations. First, rather than representing a complete distribution over the results of taking an action, which is difficult in continuous state spaces, it learns a linear model of the expected transition for each action. Second, it uses a novel strategy for compressing these linear action models, which significantly reduces space and time for learning models, and supports efficient online generation of open-loop plans. The effectiveness of these methods is demonstrated in a simulated driving domain with a 20-dimensional continuous input space.
APA
Gehring, C., Kaelbling, L.P. & Lozano-Perez, T.. (2018). Adaptable replanning with compressed linear action models for learning from demonstrations. Proceedings of The 2nd Conference on Robot Learning, in Proceedings of Machine Learning Research 87:432-442 Available from https://proceedings.mlr.press/v87/gehring18a.html.

Related Material