Coordinate Descent for SLOPE

Johan Larsson, Quentin Klopfenstein, Mathurin Massias, Jonas Wallin
Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, PMLR 206:4802-4821, 2023.

Abstract

The lasso is the most famous sparse regression and feature selection method. One reason for its popularity is the speed at which the underlying optimization problem can be solved. Sorted L-One Penalized Estimation (SLOPE) is a generalization of the lasso with appealing statistical properties. In spite of this, the method has not yet reached widespread interest. A major reason for this is that current software packages that fit SLOPE rely on algorithms that perform poorly in high dimensions. To tackle this issue, we propose a new fast algorithm to solve the SLOPE optimization problem, which combines proximal gradient descent and proximal coordinate descent steps. We provide new results on the directional derivative of the SLOPE penalty and its related SLOPE thresholding operator, as well as provide convergence guarantees for our proposed solver. In extensive benchmarks on simulated and real data, we demonstrate our method’s performance against a long list of competing algorithms.

Cite this Paper

BibTeX
@InProceedings{pmlr-v206-larsson23a, title = {Coordinate Descent for SLOPE}, author = {Larsson, Johan and Klopfenstein, Quentin and Massias, Mathurin and Wallin, Jonas}, booktitle = {Proceedings of The 26th International Conference on Artificial Intelligence and Statistics}, pages = {4802--4821}, year = {2023}, editor = {Ruiz, Francisco and Dy, Jennifer and van de Meent, Jan-Willem}, volume = {206}, series = {Proceedings of Machine Learning Research}, month = {25--27 Apr}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v206/larsson23a/larsson23a.pdf}, url = {https://proceedings.mlr.press/v206/larsson23a.html}, abstract = {The lasso is the most famous sparse regression and feature selection method. One reason for its popularity is the speed at which the underlying optimization problem can be solved. Sorted L-One Penalized Estimation (SLOPE) is a generalization of the lasso with appealing statistical properties. In spite of this, the method has not yet reached widespread interest. A major reason for this is that current software packages that fit SLOPE rely on algorithms that perform poorly in high dimensions. To tackle this issue, we propose a new fast algorithm to solve the SLOPE optimization problem, which combines proximal gradient descent and proximal coordinate descent steps. We provide new results on the directional derivative of the SLOPE penalty and its related SLOPE thresholding operator, as well as provide convergence guarantees for our proposed solver. In extensive benchmarks on simulated and real data, we demonstrate our method’s performance against a long list of competing algorithms.} }
Endnote
%0 Conference Paper %T Coordinate Descent for SLOPE %A Johan Larsson %A Quentin Klopfenstein %A Mathurin Massias %A Jonas Wallin %B Proceedings of The 26th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2023 %E Francisco Ruiz %E Jennifer Dy %E Jan-Willem van de Meent %F pmlr-v206-larsson23a %I PMLR %P 4802--4821 %U https://proceedings.mlr.press/v206/larsson23a.html %V 206 %X The lasso is the most famous sparse regression and feature selection method. One reason for its popularity is the speed at which the underlying optimization problem can be solved. Sorted L-One Penalized Estimation (SLOPE) is a generalization of the lasso with appealing statistical properties. In spite of this, the method has not yet reached widespread interest. A major reason for this is that current software packages that fit SLOPE rely on algorithms that perform poorly in high dimensions. To tackle this issue, we propose a new fast algorithm to solve the SLOPE optimization problem, which combines proximal gradient descent and proximal coordinate descent steps. We provide new results on the directional derivative of the SLOPE penalty and its related SLOPE thresholding operator, as well as provide convergence guarantees for our proposed solver. In extensive benchmarks on simulated and real data, we demonstrate our method’s performance against a long list of competing algorithms.
APA
Larsson, J., Klopfenstein, Q., Massias, M. & Wallin, J.. (2023). Coordinate Descent for SLOPE. Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 206:4802-4821 Available from https://proceedings.mlr.press/v206/larsson23a.html.

Related Material