Local Bayesian Optimization of Motor Skills

Riad Akrour, Dmitry Sorokin, Jan Peters, Gerhard Neumann
Proceedings of the 34th International Conference on Machine Learning, PMLR 70:41-50, 2017.

Abstract

Bayesian optimization is renowned for its sample efficiency but its application to higher dimensional tasks is impeded by its focus on global optimization. To scale to higher dimensional problems, we leverage the sample efficiency of Bayesian optimization in a local context. The optimization of the acquisition function is restricted to the vicinity of a Gaussian search distribution which is moved towards high value areas of the objective. The proposed information-theoretic update of the search distribution results in a Bayesian interpretation of local stochastic search: the search distribution encodes prior knowledge on the optimum’s location and is weighted at each iteration by the likelihood of this location’s optimality. We demonstrate the effectiveness of our algorithm on several benchmark objective functions as well as a continuous robotic task in which an informative prior is obtained by imitation learning.

Cite this Paper

BibTeX
@InProceedings{pmlr-v70-akrour17a, title = {Local {B}ayesian Optimization of Motor Skills}, author = {Riad Akrour and Dmitry Sorokin and Jan Peters and Gerhard Neumann}, booktitle = {Proceedings of the 34th International Conference on Machine Learning}, pages = {41--50}, year = {2017}, editor = {Precup, Doina and Teh, Yee Whye}, volume = {70}, series = {Proceedings of Machine Learning Research}, month = {06--11 Aug}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v70/akrour17a/akrour17a.pdf}, url = {https://proceedings.mlr.press/v70/akrour17a.html}, abstract = {Bayesian optimization is renowned for its sample efficiency but its application to higher dimensional tasks is impeded by its focus on global optimization. To scale to higher dimensional problems, we leverage the sample efficiency of Bayesian optimization in a local context. The optimization of the acquisition function is restricted to the vicinity of a Gaussian search distribution which is moved towards high value areas of the objective. The proposed information-theoretic update of the search distribution results in a Bayesian interpretation of local stochastic search: the search distribution encodes prior knowledge on the optimum’s location and is weighted at each iteration by the likelihood of this location’s optimality. We demonstrate the effectiveness of our algorithm on several benchmark objective functions as well as a continuous robotic task in which an informative prior is obtained by imitation learning.} }
Endnote
%0 Conference Paper %T Local Bayesian Optimization of Motor Skills %A Riad Akrour %A Dmitry Sorokin %A Jan Peters %A Gerhard Neumann %B Proceedings of the 34th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2017 %E Doina Precup %E Yee Whye Teh %F pmlr-v70-akrour17a %I PMLR %P 41--50 %U https://proceedings.mlr.press/v70/akrour17a.html %V 70 %X Bayesian optimization is renowned for its sample efficiency but its application to higher dimensional tasks is impeded by its focus on global optimization. To scale to higher dimensional problems, we leverage the sample efficiency of Bayesian optimization in a local context. The optimization of the acquisition function is restricted to the vicinity of a Gaussian search distribution which is moved towards high value areas of the objective. The proposed information-theoretic update of the search distribution results in a Bayesian interpretation of local stochastic search: the search distribution encodes prior knowledge on the optimum’s location and is weighted at each iteration by the likelihood of this location’s optimality. We demonstrate the effectiveness of our algorithm on several benchmark objective functions as well as a continuous robotic task in which an informative prior is obtained by imitation learning.
APA
Akrour, R., Sorokin, D., Peters, J. & Neumann, G.. (2017). Local Bayesian Optimization of Motor Skills. Proceedings of the 34th International Conference on Machine Learning, in Proceedings of Machine Learning Research 70:41-50 Available from https://proceedings.mlr.press/v70/akrour17a.html.

Related Material