Stochastic DCA for the Large-sum of Non-convex Functions Problem and its Application to Group Variable Selection in Classification

Hoai An Le Thi, Hoai Minh Le, Duy Nhat Phan, Bach Tran
Proceedings of the 34th International Conference on Machine Learning, PMLR 70:3394-3403, 2017.

Abstract

In this paper, we present a stochastic version of DCA (Difference of Convex functions Algorithm) to solve a class of optimization problems whose objective function is a large sum of non-convex functions and a regularization term. We consider the $\ell_{2,0}$ regularization to deal with the group variables selection. By exploiting the special structure of the problem, we propose an efficient DC decomposition for which the corresponding stochastic DCA scheme is very inexpensive: it only requires the projection of points onto balls that is explicitly computed. As an application, we applied our algorithm for the group variables selection in multiclass logistic regression. Numerical experiments on several benchmark datasets and synthetic datasets illustrate the efficiency of our algorithm and its superiority over well-known methods, with respect to classification accuracy, sparsity of solution as well as running time.

Cite this Paper

BibTeX
@InProceedings{pmlr-v70-thi17a, title = {Stochastic {DCA} for the Large-sum of Non-convex Functions Problem and its Application to Group Variable Selection in Classification}, author = {Hoai An Le Thi and Hoai Minh Le and Duy Nhat Phan and Bach Tran}, booktitle = {Proceedings of the 34th International Conference on Machine Learning}, pages = {3394--3403}, year = {2017}, editor = {Precup, Doina and Teh, Yee Whye}, volume = {70}, series = {Proceedings of Machine Learning Research}, month = {06--11 Aug}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v70/thi17a/thi17a.pdf}, url = {https://proceedings.mlr.press/v70/thi17a.html}, abstract = {In this paper, we present a stochastic version of DCA (Difference of Convex functions Algorithm) to solve a class of optimization problems whose objective function is a large sum of non-convex functions and a regularization term. We consider the $\ell_{2,0}$ regularization to deal with the group variables selection. By exploiting the special structure of the problem, we propose an efficient DC decomposition for which the corresponding stochastic DCA scheme is very inexpensive: it only requires the projection of points onto balls that is explicitly computed. As an application, we applied our algorithm for the group variables selection in multiclass logistic regression. Numerical experiments on several benchmark datasets and synthetic datasets illustrate the efficiency of our algorithm and its superiority over well-known methods, with respect to classification accuracy, sparsity of solution as well as running time.} }
Endnote
%0 Conference Paper %T Stochastic DCA for the Large-sum of Non-convex Functions Problem and its Application to Group Variable Selection in Classification %A Hoai An Le Thi %A Hoai Minh Le %A Duy Nhat Phan %A Bach Tran %B Proceedings of the 34th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2017 %E Doina Precup %E Yee Whye Teh %F pmlr-v70-thi17a %I PMLR %P 3394--3403 %U https://proceedings.mlr.press/v70/thi17a.html %V 70 %X In this paper, we present a stochastic version of DCA (Difference of Convex functions Algorithm) to solve a class of optimization problems whose objective function is a large sum of non-convex functions and a regularization term. We consider the $\ell_{2,0}$ regularization to deal with the group variables selection. By exploiting the special structure of the problem, we propose an efficient DC decomposition for which the corresponding stochastic DCA scheme is very inexpensive: it only requires the projection of points onto balls that is explicitly computed. As an application, we applied our algorithm for the group variables selection in multiclass logistic regression. Numerical experiments on several benchmark datasets and synthetic datasets illustrate the efficiency of our algorithm and its superiority over well-known methods, with respect to classification accuracy, sparsity of solution as well as running time.
APA
Thi, H.A.L., Le, H.M., Phan, D.N. & Tran, B.. (2017). Stochastic DCA for the Large-sum of Non-convex Functions Problem and its Application to Group Variable Selection in Classification. Proceedings of the 34th International Conference on Machine Learning, in Proceedings of Machine Learning Research 70:3394-3403 Available from https://proceedings.mlr.press/v70/thi17a.html.

Related Material