Exact Optimal Accelerated Complexity for Fixed-Point Iterations

Jisun Park, Ernest K Ryu
Proceedings of the 39th International Conference on Machine Learning, PMLR 162:17420-17457, 2022.

Abstract

Despite the broad use of fixed-point iterations throughout applied mathematics, the optimal convergence rate of general fixed-point problems with nonexpansive nonlinear operators has not been established. This work presents an acceleration mechanism for fixed-point iterations with nonexpansive operators, contractive operators, and nonexpansive operators satisfying a Hölder-type growth condition. We then provide matching complexity lower bounds to establish the exact optimality of the acceleration mechanisms in the nonexpansive and contractive setups. Finally, we provide experiments with CT imaging, optimal transport, and decentralized optimization to demonstrate the practical effectiveness of the acceleration mechanism.

Cite this Paper

BibTeX
@InProceedings{pmlr-v162-park22c, title = {Exact Optimal Accelerated Complexity for Fixed-Point Iterations}, author = {Park, Jisun and Ryu, Ernest K}, booktitle = {Proceedings of the 39th International Conference on Machine Learning}, pages = {17420--17457}, year = {2022}, editor = {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan}, volume = {162}, series = {Proceedings of Machine Learning Research}, month = {17--23 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v162/park22c/park22c.pdf}, url = {https://proceedings.mlr.press/v162/park22c.html}, abstract = {Despite the broad use of fixed-point iterations throughout applied mathematics, the optimal convergence rate of general fixed-point problems with nonexpansive nonlinear operators has not been established. This work presents an acceleration mechanism for fixed-point iterations with nonexpansive operators, contractive operators, and nonexpansive operators satisfying a Hölder-type growth condition. We then provide matching complexity lower bounds to establish the exact optimality of the acceleration mechanisms in the nonexpansive and contractive setups. Finally, we provide experiments with CT imaging, optimal transport, and decentralized optimization to demonstrate the practical effectiveness of the acceleration mechanism.} }
Endnote
%0 Conference Paper %T Exact Optimal Accelerated Complexity for Fixed-Point Iterations %A Jisun Park %A Ernest K Ryu %B Proceedings of the 39th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2022 %E Kamalika Chaudhuri %E Stefanie Jegelka %E Le Song %E Csaba Szepesvari %E Gang Niu %E Sivan Sabato %F pmlr-v162-park22c %I PMLR %P 17420--17457 %U https://proceedings.mlr.press/v162/park22c.html %V 162 %X Despite the broad use of fixed-point iterations throughout applied mathematics, the optimal convergence rate of general fixed-point problems with nonexpansive nonlinear operators has not been established. This work presents an acceleration mechanism for fixed-point iterations with nonexpansive operators, contractive operators, and nonexpansive operators satisfying a Hölder-type growth condition. We then provide matching complexity lower bounds to establish the exact optimality of the acceleration mechanisms in the nonexpansive and contractive setups. Finally, we provide experiments with CT imaging, optimal transport, and decentralized optimization to demonstrate the practical effectiveness of the acceleration mechanism.
APA
Park, J. & Ryu, E.K.. (2022). Exact Optimal Accelerated Complexity for Fixed-Point Iterations. Proceedings of the 39th International Conference on Machine Learning, in Proceedings of Machine Learning Research 162:17420-17457 Available from https://proceedings.mlr.press/v162/park22c.html.

Related Material