OpReg-Boost: Learning to Accelerate Online Algorithms with Operator Regression

Nicola Bastianello, Andrea Simonetto, Emiliano Dall’Anese
Proceedings of The 4th Annual Learning for Dynamics and Control Conference, PMLR 168:138-152, 2022.

Abstract

This paper presents a new regularization approach – termed OpReg-Boost – to boost the convergence of online optimization and learning algorithms. In particular, the paper considers online algorithms for optimization problems with a time-varying (weakly) convex composite cost. For a given online algorithm, OpReg-Boost learns the closest algorithmic map that yields linear convergence; to this end, the learning procedure hinges on the concept of operator regression. We show how to formalize the operator regression problem and propose a computationally-efficient Peaceman-Rachford solver that exploits a closed-form solution of simple quadratically-constrained quadratic programs (QCQPs). Simulation results showcase the superior properties of OpReg-Boost w.r.t. the more classical forward-backward algorithm, FISTA, and Anderson acceleration.

Cite this Paper

BibTeX
@InProceedings{pmlr-v168-bastianello22a, title = {OpReg-Boost: Learning to Accelerate Online Algorithms with Operator Regression}, author = {Bastianello, Nicola and Simonetto, Andrea and Dall'Anese, Emiliano}, booktitle = {Proceedings of The 4th Annual Learning for Dynamics and Control Conference}, pages = {138--152}, year = {2022}, editor = {Firoozi, Roya and Mehr, Negar and Yel, Esen and Antonova, Rika and Bohg, Jeannette and Schwager, Mac and Kochenderfer, Mykel}, volume = {168}, series = {Proceedings of Machine Learning Research}, month = {23--24 Jun}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v168/bastianello22a/bastianello22a.pdf}, url = {https://proceedings.mlr.press/v168/bastianello22a.html}, abstract = {This paper presents a new regularization approach – termed OpReg-Boost – to boost the convergence of online optimization and learning algorithms. In particular, the paper considers online algorithms for optimization problems with a time-varying (weakly) convex composite cost. For a given online algorithm, OpReg-Boost learns the closest algorithmic map that yields linear convergence; to this end, the learning procedure hinges on the concept of operator regression. We show how to formalize the operator regression problem and propose a computationally-efficient Peaceman-Rachford solver that exploits a closed-form solution of simple quadratically-constrained quadratic programs (QCQPs). Simulation results showcase the superior properties of OpReg-Boost w.r.t. the more classical forward-backward algorithm, FISTA, and Anderson acceleration.} }
Endnote
%0 Conference Paper %T OpReg-Boost: Learning to Accelerate Online Algorithms with Operator Regression %A Nicola Bastianello %A Andrea Simonetto %A Emiliano Dall’Anese %B Proceedings of The 4th Annual Learning for Dynamics and Control Conference %C Proceedings of Machine Learning Research %D 2022 %E Roya Firoozi %E Negar Mehr %E Esen Yel %E Rika Antonova %E Jeannette Bohg %E Mac Schwager %E Mykel Kochenderfer %F pmlr-v168-bastianello22a %I PMLR %P 138--152 %U https://proceedings.mlr.press/v168/bastianello22a.html %V 168 %X This paper presents a new regularization approach – termed OpReg-Boost – to boost the convergence of online optimization and learning algorithms. In particular, the paper considers online algorithms for optimization problems with a time-varying (weakly) convex composite cost. For a given online algorithm, OpReg-Boost learns the closest algorithmic map that yields linear convergence; to this end, the learning procedure hinges on the concept of operator regression. We show how to formalize the operator regression problem and propose a computationally-efficient Peaceman-Rachford solver that exploits a closed-form solution of simple quadratically-constrained quadratic programs (QCQPs). Simulation results showcase the superior properties of OpReg-Boost w.r.t. the more classical forward-backward algorithm, FISTA, and Anderson acceleration.
APA
Bastianello, N., Simonetto, A. & Dall’Anese, E.. (2022). OpReg-Boost: Learning to Accelerate Online Algorithms with Operator Regression. Proceedings of The 4th Annual Learning for Dynamics and Control Conference, in Proceedings of Machine Learning Research 168:138-152 Available from https://proceedings.mlr.press/v168/bastianello22a.html.

Related Material