Active Learning of Parameterized Skills

Bruno Da Silva, George Konidaris, Andrew Barto
Proceedings of the 31st International Conference on Machine Learning, PMLR 32(2):1737-1745, 2014.

Abstract

We introduce a method for actively learning parameterized skills. Parameterized skills are flexible behaviors that can solve any task drawn from a distribution of parameterized reinforcement learning problems. Approaches to learning such skills have been proposed, but limited attention has been given to identifying which training tasks allow for rapid skill acquisition. We construct a non-parametric Bayesian model of skill performance and derive analytical expressions for a novel acquisition criterion capable of identifying tasks that maximize expected improvement in skill performance. We also introduce a spatiotemporal kernel tailored for non-stationary skill performance models. The proposed method is agnostic to policy and skill representation and scales independently of task dimensionality. We evaluate it on a non-linear simulated catapult control problem over arbitrarily mountainous terrains.

Cite this Paper

BibTeX
@InProceedings{pmlr-v32-silva14, title = {Active Learning of Parameterized Skills}, author = {Silva, Bruno Da and Konidaris, George and Barto, Andrew}, booktitle = {Proceedings of the 31st International Conference on Machine Learning}, pages = {1737--1745}, year = {2014}, editor = {Xing, Eric P. and Jebara, Tony}, volume = {32}, number = {2}, series = {Proceedings of Machine Learning Research}, address = {Bejing, China}, month = {22--24 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v32/silva14.pdf}, url = {https://proceedings.mlr.press/v32/silva14.html}, abstract = {We introduce a method for actively learning parameterized skills. Parameterized skills are flexible behaviors that can solve any task drawn from a distribution of parameterized reinforcement learning problems. Approaches to learning such skills have been proposed, but limited attention has been given to identifying which training tasks allow for rapid skill acquisition. We construct a non-parametric Bayesian model of skill performance and derive analytical expressions for a novel acquisition criterion capable of identifying tasks that maximize expected improvement in skill performance. We also introduce a spatiotemporal kernel tailored for non-stationary skill performance models. The proposed method is agnostic to policy and skill representation and scales independently of task dimensionality. We evaluate it on a non-linear simulated catapult control problem over arbitrarily mountainous terrains.} }
Endnote
%0 Conference Paper %T Active Learning of Parameterized Skills %A Bruno Da Silva %A George Konidaris %A Andrew Barto %B Proceedings of the 31st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2014 %E Eric P. Xing %E Tony Jebara %F pmlr-v32-silva14 %I PMLR %P 1737--1745 %U https://proceedings.mlr.press/v32/silva14.html %V 32 %N 2 %X We introduce a method for actively learning parameterized skills. Parameterized skills are flexible behaviors that can solve any task drawn from a distribution of parameterized reinforcement learning problems. Approaches to learning such skills have been proposed, but limited attention has been given to identifying which training tasks allow for rapid skill acquisition. We construct a non-parametric Bayesian model of skill performance and derive analytical expressions for a novel acquisition criterion capable of identifying tasks that maximize expected improvement in skill performance. We also introduce a spatiotemporal kernel tailored for non-stationary skill performance models. The proposed method is agnostic to policy and skill representation and scales independently of task dimensionality. We evaluate it on a non-linear simulated catapult control problem over arbitrarily mountainous terrains.
RIS
TY - CPAPER TI - Active Learning of Parameterized Skills AU - Bruno Da Silva AU - George Konidaris AU - Andrew Barto BT - Proceedings of the 31st International Conference on Machine Learning DA - 2014/06/18 ED - Eric P. Xing ED - Tony Jebara ID - pmlr-v32-silva14 PB - PMLR DP - Proceedings of Machine Learning Research VL - 32 IS - 2 SP - 1737 EP - 1745 L1 - http://proceedings.mlr.press/v32/silva14.pdf UR - https://proceedings.mlr.press/v32/silva14.html AB - We introduce a method for actively learning parameterized skills. Parameterized skills are flexible behaviors that can solve any task drawn from a distribution of parameterized reinforcement learning problems. Approaches to learning such skills have been proposed, but limited attention has been given to identifying which training tasks allow for rapid skill acquisition. We construct a non-parametric Bayesian model of skill performance and derive analytical expressions for a novel acquisition criterion capable of identifying tasks that maximize expected improvement in skill performance. We also introduce a spatiotemporal kernel tailored for non-stationary skill performance models. The proposed method is agnostic to policy and skill representation and scales independently of task dimensionality. We evaluate it on a non-linear simulated catapult control problem over arbitrarily mountainous terrains. ER -
APA
Silva, B.D., Konidaris, G. & Barto, A.. (2014). Active Learning of Parameterized Skills. Proceedings of the 31st International Conference on Machine Learning, in Proceedings of Machine Learning Research 32(2):1737-1745 Available from https://proceedings.mlr.press/v32/silva14.html.

Related Material