The Impact of the Geometric Properties of the Constraint Set in Safe Optimization with Bandit Feedback

Spencer Hutchinson, Berkay Turan, Mahnoosh Alizadeh
Proceedings of The 5th Annual Learning for Dynamics and Control Conference, PMLR 211:497-508, 2023.

Abstract

We consider a safe optimization problem with bandit feedback in which an agent sequentially chooses actions and observes responses from the environment, with the goal of maximizing an arbitrary function of the response while respecting stage-wise constraints. We propose an algorithm for this problem, and study how the geometric properties of the constraint set impact the regret of the algorithm. In order to do so, we introduce the notion of the sharpness of a particular constraint set, which characterizes the difficulty of performing learning within the constraint set in an uncertain setting. This concept of sharpness allows us to identify the class of constraint sets for which the proposed algorithm is guaranteed to enjoy sublinear regret. Simulation results for this algorithm support the sublinear regret bound and provide empirical evidence that the sharpness of the constraint set impacts the performance of the algorithm.

Cite this Paper

BibTeX
@InProceedings{pmlr-v211-hutchinson23a, title = {The Impact of the Geometric Properties of the Constraint Set in Safe Optimization with Bandit Feedback}, author = {Hutchinson, Spencer and Turan, Berkay and Alizadeh, Mahnoosh}, booktitle = {Proceedings of The 5th Annual Learning for Dynamics and Control Conference}, pages = {497--508}, year = {2023}, editor = {Matni, Nikolai and Morari, Manfred and Pappas, George J.}, volume = {211}, series = {Proceedings of Machine Learning Research}, month = {15--16 Jun}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v211/hutchinson23a/hutchinson23a.pdf}, url = {https://proceedings.mlr.press/v211/hutchinson23a.html}, abstract = {We consider a safe optimization problem with bandit feedback in which an agent sequentially chooses actions and observes responses from the environment, with the goal of maximizing an arbitrary function of the response while respecting stage-wise constraints. We propose an algorithm for this problem, and study how the geometric properties of the constraint set impact the regret of the algorithm. In order to do so, we introduce the notion of the sharpness of a particular constraint set, which characterizes the difficulty of performing learning within the constraint set in an uncertain setting. This concept of sharpness allows us to identify the class of constraint sets for which the proposed algorithm is guaranteed to enjoy sublinear regret. Simulation results for this algorithm support the sublinear regret bound and provide empirical evidence that the sharpness of the constraint set impacts the performance of the algorithm.} }
Endnote
%0 Conference Paper %T The Impact of the Geometric Properties of the Constraint Set in Safe Optimization with Bandit Feedback %A Spencer Hutchinson %A Berkay Turan %A Mahnoosh Alizadeh %B Proceedings of The 5th Annual Learning for Dynamics and Control Conference %C Proceedings of Machine Learning Research %D 2023 %E Nikolai Matni %E Manfred Morari %E George J. Pappas %F pmlr-v211-hutchinson23a %I PMLR %P 497--508 %U https://proceedings.mlr.press/v211/hutchinson23a.html %V 211 %X We consider a safe optimization problem with bandit feedback in which an agent sequentially chooses actions and observes responses from the environment, with the goal of maximizing an arbitrary function of the response while respecting stage-wise constraints. We propose an algorithm for this problem, and study how the geometric properties of the constraint set impact the regret of the algorithm. In order to do so, we introduce the notion of the sharpness of a particular constraint set, which characterizes the difficulty of performing learning within the constraint set in an uncertain setting. This concept of sharpness allows us to identify the class of constraint sets for which the proposed algorithm is guaranteed to enjoy sublinear regret. Simulation results for this algorithm support the sublinear regret bound and provide empirical evidence that the sharpness of the constraint set impacts the performance of the algorithm.
APA
Hutchinson, S., Turan, B. & Alizadeh, M.. (2023). The Impact of the Geometric Properties of the Constraint Set in Safe Optimization with Bandit Feedback. Proceedings of The 5th Annual Learning for Dynamics and Control Conference, in Proceedings of Machine Learning Research 211:497-508 Available from https://proceedings.mlr.press/v211/hutchinson23a.html.

Related Material