High-dimensional Joint Sparsity Random Effects Model for Multi-task Learning

Krishnakumar Balasubramanian, Kai Yu, Tong Zhang
Proceedings of the 29th Conference on Uncertainty in Artificial Intelligence, PMLR R11:282-291, 2013.

Abstract

Joint sparsity regularization in multi-task learn- ing has attracted much attention in recent years. The traditional convex formulation employs the group Lasso relaxation to achieve joint sparsity across tasks. Although this approach leads to a simple convex formulation, it suffers from sev- eral issues due to the looseness of the relax- ation. To remedy this problem, we view jointly sparse multi-task learning as a specialized ran- dom effects model, and derive a convex relax- ation approach that involves two steps. The first step learns the covariance matrix of the coef- ficients using a convex formulation which we refer to as sparse covariance coding; the sec- ond step solves a ridge regression problem with a sparse quadratic regularizer based on the co- variance matrix obtained in the first step. It is shown that this approach produces an asymptot- ically optimal quadratic regularizer in the mul- titask learning setting when the number of tasks approaches infinity. Experimental results demon- strate that the convex formulation obtained via the proposed model significantly outperforms group Lasso (and related multi-stage formula- tions).

Cite this Paper

BibTeX
@InProceedings{pmlr-vR11-balasubramanian13a, title = {High-dimensional Joint Sparsity Random Effects Model for Multi-task Learning}, author = {Balasubramanian, Krishnakumar and Yu, Kai and Zhang, Tong}, booktitle = {Proceedings of the 29th Conference on Uncertainty in Artificial Intelligence}, pages = {282--291}, year = {2013}, editor = {Nicholson, Ann and Smyth, Padhraic}, volume = {R11}, series = {Proceedings of Machine Learning Research}, month = {12--14 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r11/main/assets/balasubramanian13a/balasubramanian13a.pdf}, url = {https://proceedings.mlr.press/r11/balasubramanian13a.html}, abstract = {Joint sparsity regularization in multi-task learn- ing has attracted much attention in recent years. The traditional convex formulation employs the group Lasso relaxation to achieve joint sparsity across tasks. Although this approach leads to a simple convex formulation, it suffers from sev- eral issues due to the looseness of the relax- ation. To remedy this problem, we view jointly sparse multi-task learning as a specialized ran- dom effects model, and derive a convex relax- ation approach that involves two steps. The first step learns the covariance matrix of the coef- ficients using a convex formulation which we refer to as sparse covariance coding; the sec- ond step solves a ridge regression problem with a sparse quadratic regularizer based on the co- variance matrix obtained in the first step. It is shown that this approach produces an asymptot- ically optimal quadratic regularizer in the mul- titask learning setting when the number of tasks approaches infinity. Experimental results demon- strate that the convex formulation obtained via the proposed model significantly outperforms group Lasso (and related multi-stage formula- tions).}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T High-dimensional Joint Sparsity Random Effects Model for Multi-task Learning %A Krishnakumar Balasubramanian %A Kai Yu %A Tong Zhang %B Proceedings of the 29th Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2013 %E Ann Nicholson %E Padhraic Smyth %F pmlr-vR11-balasubramanian13a %I PMLR %P 282--291 %U https://proceedings.mlr.press/r11/balasubramanian13a.html %V R11 %X Joint sparsity regularization in multi-task learn- ing has attracted much attention in recent years. The traditional convex formulation employs the group Lasso relaxation to achieve joint sparsity across tasks. Although this approach leads to a simple convex formulation, it suffers from sev- eral issues due to the looseness of the relax- ation. To remedy this problem, we view jointly sparse multi-task learning as a specialized ran- dom effects model, and derive a convex relax- ation approach that involves two steps. The first step learns the covariance matrix of the coef- ficients using a convex formulation which we refer to as sparse covariance coding; the sec- ond step solves a ridge regression problem with a sparse quadratic regularizer based on the co- variance matrix obtained in the first step. It is shown that this approach produces an asymptot- ically optimal quadratic regularizer in the mul- titask learning setting when the number of tasks approaches infinity. Experimental results demon- strate that the convex formulation obtained via the proposed model significantly outperforms group Lasso (and related multi-stage formula- tions). %Z Reissued by PMLR on 04 October 2026.
APA
Balasubramanian, K., Yu, K. & Zhang, T.. (2013). High-dimensional Joint Sparsity Random Effects Model for Multi-task Learning. Proceedings of the 29th Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R11:282-291 Available from https://proceedings.mlr.press/r11/balasubramanian13a.html. Reissued by PMLR on 04 October 2026.

Related Material