A General Analysis of the Convergence of ADMM

[edit]

Robert Nishihara, Laurent Lessard, Ben Recht, Andrew Packard, Michael Jordan ;
Proceedings of the 32nd International Conference on Machine Learning, PMLR 37:343-352, 2015.

Abstract

We provide a new proof of the linear convergence of the alternating direction method of multipliers (ADMM) when one of the objective terms is strongly convex. Our proof is based on a framework for analyzing optimization algorithms introduced in Lessard et al. (2014), reducing algorithm convergence to verifying the stability of a dynamical system. This approach generalizes a number of existing results and obviates any assumptions about specific choices of algorithm parameters. On a numerical example, we demonstrate that minimizing the derived bound on the convergence rate provides a practical approach to selecting algorithm parameters for particular ADMM instances. We complement our upper bound by constructing a nearly-matching lower bound on the worst-case rate of convergence.

Related Material