Safe non-smooth black-box optimization with application to policy search

Ilnura Usmanova, Andreas Krause, Maryam Kamgarpour
Proceedings of the 2nd Conference on Learning for Dynamics and Control, PMLR 120:980-989, 2020.

Abstract

For safety-critical black-box optimization tasks, observations of the constraints and the objective are often noisy and available only for the feasible points. We propose an approach based on log barriers to find a local solution of a non-convex non-smooth black-box optimization problem $\min f^0(x)$ subject to $f^i(x)\leq 0, i = 1,\ldots, m$, at the same time, guaranteeing constraint satisfaction while learning with high probability. Our proposed algorithm exploits noisy observations to iteratively improve on an initial safe point until convergence. We derive the convergence rate and prove safety of our algorithm. We demonstrate its performance in an application to an iterative control design problem.

Cite this Paper

BibTeX
@InProceedings{pmlr-v120-usmanova20a, title = {Safe non-smooth black-box optimization with application to policy search}, author = {Usmanova, Ilnura and Krause, Andreas and Kamgarpour, Maryam}, booktitle = {Proceedings of the 2nd Conference on Learning for Dynamics and Control}, pages = {980--989}, year = {2020}, editor = {Bayen, Alexandre M. and Jadbabaie, Ali and Pappas, George and Parrilo, Pablo A. and Recht, Benjamin and Tomlin, Claire and Zeilinger, Melanie}, volume = {120}, series = {Proceedings of Machine Learning Research}, month = {10--11 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v120/usmanova20a/usmanova20a.pdf}, url = {https://proceedings.mlr.press/v120/usmanova20a.html}, abstract = {For safety-critical black-box optimization tasks, observations of the constraints and the objective are often noisy and available only for the feasible points. We propose an approach based on log barriers to find a local solution of a non-convex non-smooth black-box optimization problem $\min f^0(x)$ subject to $f^i(x)\leq 0, i = 1,\ldots, m$, at the same time, guaranteeing constraint satisfaction while learning with high probability. Our proposed algorithm exploits noisy observations to iteratively improve on an initial safe point until convergence. We derive the convergence rate and prove safety of our algorithm. We demonstrate its performance in an application to an iterative control design problem.} }
Endnote
%0 Conference Paper %T Safe non-smooth black-box optimization with application to policy search %A Ilnura Usmanova %A Andreas Krause %A Maryam Kamgarpour %B Proceedings of the 2nd Conference on Learning for Dynamics and Control %C Proceedings of Machine Learning Research %D 2020 %E Alexandre M. Bayen %E Ali Jadbabaie %E George Pappas %E Pablo A. Parrilo %E Benjamin Recht %E Claire Tomlin %E Melanie Zeilinger %F pmlr-v120-usmanova20a %I PMLR %P 980--989 %U https://proceedings.mlr.press/v120/usmanova20a.html %V 120 %X For safety-critical black-box optimization tasks, observations of the constraints and the objective are often noisy and available only for the feasible points. We propose an approach based on log barriers to find a local solution of a non-convex non-smooth black-box optimization problem $\min f^0(x)$ subject to $f^i(x)\leq 0, i = 1,\ldots, m$, at the same time, guaranteeing constraint satisfaction while learning with high probability. Our proposed algorithm exploits noisy observations to iteratively improve on an initial safe point until convergence. We derive the convergence rate and prove safety of our algorithm. We demonstrate its performance in an application to an iterative control design problem.
APA
Usmanova, I., Krause, A. & Kamgarpour, M.. (2020). Safe non-smooth black-box optimization with application to policy search. Proceedings of the 2nd Conference on Learning for Dynamics and Control, in Proceedings of Machine Learning Research 120:980-989 Available from https://proceedings.mlr.press/v120/usmanova20a.html.

Related Material