Accelerated Coordinate Descent with Adaptive Coordinate Frequencies

Tobias Glasmachers, Urun Dogan
Proceedings of the 5th Asian Conference on Machine Learning, PMLR 29:72-86, 2013.

Abstract

Coordinate descent (CD) algorithms have become the method of choice for solving a number of machine learning tasks. They are particularly popular for training linear models, including linear support vector machine classification, LASSO regression, and logistic regression. We propose an extension of the CD algorithm, called the adaptive coordinate frequencies (ACF) method. This modified CD scheme does not treat all coordinates equally, in that it does not pick all coordinates equally often for optimization. Instead the relative frequencies of coordinates are subject to online adaptation. The resulting optimization scheme can result in significant speed-ups. We demonstrate the usefulness of our approach on a number of large scale machine learning problems.

Cite this Paper

BibTeX
@InProceedings{pmlr-v29-Glasmachers13, title = {Accelerated Coordinate Descent with Adaptive Coordinate Frequencies}, author = {Glasmachers, Tobias and Dogan, Urun}, booktitle = {Proceedings of the 5th Asian Conference on Machine Learning}, pages = {72--86}, year = {2013}, editor = {Ong, Cheng Soon and Ho, Tu Bao}, volume = {29}, series = {Proceedings of Machine Learning Research}, address = {Australian National University, Canberra, Australia}, month = {13--15 Nov}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v29/Glasmachers13.pdf}, url = {https://proceedings.mlr.press/v29/Glasmachers13.html}, abstract = {Coordinate descent (CD) algorithms have become the method of choice for solving a number of machine learning tasks. They are particularly popular for training linear models, including linear support vector machine classification, LASSO regression, and logistic regression. We propose an extension of the CD algorithm, called the adaptive coordinate frequencies (ACF) method. This modified CD scheme does not treat all coordinates equally, in that it does not pick all coordinates equally often for optimization. Instead the relative frequencies of coordinates are subject to online adaptation. The resulting optimization scheme can result in significant speed-ups. We demonstrate the usefulness of our approach on a number of large scale machine learning problems.} }
Endnote
%0 Conference Paper %T Accelerated Coordinate Descent with Adaptive Coordinate Frequencies %A Tobias Glasmachers %A Urun Dogan %B Proceedings of the 5th Asian Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2013 %E Cheng Soon Ong %E Tu Bao Ho %F pmlr-v29-Glasmachers13 %I PMLR %P 72--86 %U https://proceedings.mlr.press/v29/Glasmachers13.html %V 29 %X Coordinate descent (CD) algorithms have become the method of choice for solving a number of machine learning tasks. They are particularly popular for training linear models, including linear support vector machine classification, LASSO regression, and logistic regression. We propose an extension of the CD algorithm, called the adaptive coordinate frequencies (ACF) method. This modified CD scheme does not treat all coordinates equally, in that it does not pick all coordinates equally often for optimization. Instead the relative frequencies of coordinates are subject to online adaptation. The resulting optimization scheme can result in significant speed-ups. We demonstrate the usefulness of our approach on a number of large scale machine learning problems.
RIS
TY - CPAPER TI - Accelerated Coordinate Descent with Adaptive Coordinate Frequencies AU - Tobias Glasmachers AU - Urun Dogan BT - Proceedings of the 5th Asian Conference on Machine Learning DA - 2013/10/21 ED - Cheng Soon Ong ED - Tu Bao Ho ID - pmlr-v29-Glasmachers13 PB - PMLR DP - Proceedings of Machine Learning Research VL - 29 SP - 72 EP - 86 L1 - http://proceedings.mlr.press/v29/Glasmachers13.pdf UR - https://proceedings.mlr.press/v29/Glasmachers13.html AB - Coordinate descent (CD) algorithms have become the method of choice for solving a number of machine learning tasks. They are particularly popular for training linear models, including linear support vector machine classification, LASSO regression, and logistic regression. We propose an extension of the CD algorithm, called the adaptive coordinate frequencies (ACF) method. This modified CD scheme does not treat all coordinates equally, in that it does not pick all coordinates equally often for optimization. Instead the relative frequencies of coordinates are subject to online adaptation. The resulting optimization scheme can result in significant speed-ups. We demonstrate the usefulness of our approach on a number of large scale machine learning problems. ER -
APA
Glasmachers, T. & Dogan, U.. (2013). Accelerated Coordinate Descent with Adaptive Coordinate Frequencies. Proceedings of the 5th Asian Conference on Machine Learning, in Proceedings of Machine Learning Research 29:72-86 Available from https://proceedings.mlr.press/v29/Glasmachers13.html.

Related Material