Optimization with First-Order Surrogate Functions

Julien Mairal
Proceedings of the 30th International Conference on Machine Learning, PMLR 28(3):783-791, 2013.

Abstract

In this paper, we study optimization methods consisting of iteratively minimizing surrogates of an objective function. By proposing several algorithmic variants and simple convergence analyses, we make two main contributions. First, we provide a unified viewpoint for several first-order optimization techniques such as accelerated proximal gradient, block coordinate descent, or Frank-Wolfe algorithms. Second, we introduce a new incremental scheme that experimentally matches or outperforms state-of-the-art solvers for large-scale optimization problems typically arising in machine learning.

Cite this Paper

BibTeX
@InProceedings{pmlr-v28-mairal13, title = {Optimization with First-Order Surrogate Functions}, author = {Mairal, Julien}, booktitle = {Proceedings of the 30th International Conference on Machine Learning}, pages = {783--791}, year = {2013}, editor = {Dasgupta, Sanjoy and McAllester, David}, volume = {28}, number = {3}, series = {Proceedings of Machine Learning Research}, address = {Atlanta, Georgia, USA}, month = {17--19 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v28/mairal13.pdf}, url = {https://proceedings.mlr.press/v28/mairal13.html}, abstract = {In this paper, we study optimization methods consisting of iteratively minimizing surrogates of an objective function. By proposing several algorithmic variants and simple convergence analyses, we make two main contributions. First, we provide a unified viewpoint for several first-order optimization techniques such as accelerated proximal gradient, block coordinate descent, or Frank-Wolfe algorithms. Second, we introduce a new incremental scheme that experimentally matches or outperforms state-of-the-art solvers for large-scale optimization problems typically arising in machine learning.} }
Endnote
%0 Conference Paper %T Optimization with First-Order Surrogate Functions %A Julien Mairal %B Proceedings of the 30th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2013 %E Sanjoy Dasgupta %E David McAllester %F pmlr-v28-mairal13 %I PMLR %P 783--791 %U https://proceedings.mlr.press/v28/mairal13.html %V 28 %N 3 %X In this paper, we study optimization methods consisting of iteratively minimizing surrogates of an objective function. By proposing several algorithmic variants and simple convergence analyses, we make two main contributions. First, we provide a unified viewpoint for several first-order optimization techniques such as accelerated proximal gradient, block coordinate descent, or Frank-Wolfe algorithms. Second, we introduce a new incremental scheme that experimentally matches or outperforms state-of-the-art solvers for large-scale optimization problems typically arising in machine learning.
RIS
TY - CPAPER TI - Optimization with First-Order Surrogate Functions AU - Julien Mairal BT - Proceedings of the 30th International Conference on Machine Learning DA - 2013/05/26 ED - Sanjoy Dasgupta ED - David McAllester ID - pmlr-v28-mairal13 PB - PMLR DP - Proceedings of Machine Learning Research VL - 28 IS - 3 SP - 783 EP - 791 L1 - http://proceedings.mlr.press/v28/mairal13.pdf UR - https://proceedings.mlr.press/v28/mairal13.html AB - In this paper, we study optimization methods consisting of iteratively minimizing surrogates of an objective function. By proposing several algorithmic variants and simple convergence analyses, we make two main contributions. First, we provide a unified viewpoint for several first-order optimization techniques such as accelerated proximal gradient, block coordinate descent, or Frank-Wolfe algorithms. Second, we introduce a new incremental scheme that experimentally matches or outperforms state-of-the-art solvers for large-scale optimization problems typically arising in machine learning. ER -
APA
Mairal, J.. (2013). Optimization with First-Order Surrogate Functions. Proceedings of the 30th International Conference on Machine Learning, in Proceedings of Machine Learning Research 28(3):783-791 Available from https://proceedings.mlr.press/v28/mairal13.html.

Related Material