Adaptive ADMM with Spectral Penalty Parameter Selection

Zheng Xu, Mario Figueiredo, Tom Goldstein
Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, PMLR 54:718-727, 2017.

Abstract

The alternating direction method of multipliers (ADMM) is a versatile tool for solving a wide range of constrained optimization problems. However, its performance is highly sensitive to a penalty parameter, making ADMM often unreliable and hard to automate for a non-expert user. We tackle this weakness of ADMM by proposing a method that adaptively tunes the penalty parameter to achieve fast convergence. The resulting adaptive ADMM (AADMM) algorithm, inspired by the successful Barzilai-Borwein spectral method for gradient descent, yields fast convergence and relative insensitivity to the initial stepsize and problem scaling.

Cite this Paper

BibTeX
@InProceedings{pmlr-v54-xu17a, title = {{Adaptive ADMM with Spectral Penalty Parameter Selection}}, author = {Xu, Zheng and Figueiredo, Mario and Goldstein, Tom}, booktitle = {Proceedings of the 20th International Conference on Artificial Intelligence and Statistics}, pages = {718--727}, year = {2017}, editor = {Singh, Aarti and Zhu, Jerry}, volume = {54}, series = {Proceedings of Machine Learning Research}, month = {20--22 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v54/xu17a/xu17a.pdf}, url = {https://proceedings.mlr.press/v54/xu17a.html}, abstract = {The alternating direction method of multipliers (ADMM) is a versatile tool for solving a wide range of constrained optimization problems. However, its performance is highly sensitive to a penalty parameter, making ADMM often unreliable and hard to automate for a non-expert user. We tackle this weakness of ADMM by proposing a method that adaptively tunes the penalty parameter to achieve fast convergence. The resulting adaptive ADMM (AADMM) algorithm, inspired by the successful Barzilai-Borwein spectral method for gradient descent, yields fast convergence and relative insensitivity to the initial stepsize and problem scaling.} }
Endnote
%0 Conference Paper %T Adaptive ADMM with Spectral Penalty Parameter Selection %A Zheng Xu %A Mario Figueiredo %A Tom Goldstein %B Proceedings of the 20th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2017 %E Aarti Singh %E Jerry Zhu %F pmlr-v54-xu17a %I PMLR %P 718--727 %U https://proceedings.mlr.press/v54/xu17a.html %V 54 %X The alternating direction method of multipliers (ADMM) is a versatile tool for solving a wide range of constrained optimization problems. However, its performance is highly sensitive to a penalty parameter, making ADMM often unreliable and hard to automate for a non-expert user. We tackle this weakness of ADMM by proposing a method that adaptively tunes the penalty parameter to achieve fast convergence. The resulting adaptive ADMM (AADMM) algorithm, inspired by the successful Barzilai-Borwein spectral method for gradient descent, yields fast convergence and relative insensitivity to the initial stepsize and problem scaling.
APA
Xu, Z., Figueiredo, M. & Goldstein, T.. (2017). Adaptive ADMM with Spectral Penalty Parameter Selection. Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 54:718-727 Available from https://proceedings.mlr.press/v54/xu17a.html.

Related Material