Adaptive ADMM with Spectral Penalty Parameter Selection


Zheng Xu, Mario Figueiredo, Tom Goldstein ;
Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, PMLR 54:718-727, 2017.


The alternating direction method of multipliers (ADMM) is a versatile tool for solving a wide range of constrained optimization problems. However, its performance is highly sensitive to a penalty parameter, making ADMM often unreliable and hard to automate for a non-expert user. We tackle this weakness of ADMM by proposing a method that adaptively tunes the penalty parameter to achieve fast convergence. The resulting adaptive ADMM (AADMM) algorithm, inspired by the successful Barzilai-Borwein spectral method for gradient descent, yields fast convergence and relative insensitivity to the initial stepsize and problem scaling.

Related Material