Block Regularized Lasso for Multivariate Multi-Response Linear Regression

Weiguang Wang, Yingbin Liang, Eric Xing
Proceedings of the Sixteenth International Conference on Artificial Intelligence and Statistics, PMLR 31:608-617, 2013.

Abstract

The multivariate multi-response (MVMR) linear regression problem is investigated, in which design matrices can be distributed differently across K linear regressions. The support union of K p-dimensional regression vectors are recovered via block regularized Lasso which uses the l_1/l_2 norm for regression vectors across K tasks. Sufficient and necessary conditions to guarantee successful recovery of the support union are characterized. More specifically, it is shown that under certain conditions on the distributions of design matrices, if n > c_p1 ψ(B^*,Σ^(1:K))\log(p-s) where c_p1 is a constant and s is the size of the support set, then the l_1/l_2 regularized Lasso correctly recovers the support union; and if n < c_p2 ψ(B^*,Σ^(1:K))\log(p-s) where c_p2 is a constant, then the l_1/l_2 regularized Lasso fails to recover the support union. In particular, ψ(B^*,Σ^(1:K)) captures the sparsity of K regression vectors and the statistical properties of the design matrices. Numerical results are provided to demonstrate the advantages of joint support union recovery using multi-task Lasso problem over studying each problem individually.

Cite this Paper

BibTeX
@InProceedings{pmlr-v31-wang13c, title = {Block Regularized Lasso for Multivariate Multi-Response Linear Regression}, author = {Wang, Weiguang and Liang, Yingbin and Xing, Eric}, booktitle = {Proceedings of the Sixteenth International Conference on Artificial Intelligence and Statistics}, pages = {608--617}, year = {2013}, editor = {Carvalho, Carlos M. and Ravikumar, Pradeep}, volume = {31}, series = {Proceedings of Machine Learning Research}, address = {Scottsdale, Arizona, USA}, month = {29 Apr--01 May}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v31/wang13c.pdf}, url = {https://proceedings.mlr.press/v31/wang13c.html}, abstract = {The multivariate multi-response (MVMR) linear regression problem is investigated, in which design matrices can be distributed differently across K linear regressions. The support union of K p-dimensional regression vectors are recovered via block regularized Lasso which uses the l_1/l_2 norm for regression vectors across K tasks. Sufficient and necessary conditions to guarantee successful recovery of the support union are characterized. More specifically, it is shown that under certain conditions on the distributions of design matrices, if n > c_p1 ψ(B^*,Σ^(1:K))\log(p-s) where c_p1 is a constant and s is the size of the support set, then the l_1/l_2 regularized Lasso correctly recovers the support union; and if n < c_p2 ψ(B^*,Σ^(1:K))\log(p-s) where c_p2 is a constant, then the l_1/l_2 regularized Lasso fails to recover the support union. In particular, ψ(B^*,Σ^(1:K)) captures the sparsity of K regression vectors and the statistical properties of the design matrices. Numerical results are provided to demonstrate the advantages of joint support union recovery using multi-task Lasso problem over studying each problem individually.} }
Endnote
%0 Conference Paper %T Block Regularized Lasso for Multivariate Multi-Response Linear Regression %A Weiguang Wang %A Yingbin Liang %A Eric Xing %B Proceedings of the Sixteenth International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2013 %E Carlos M. Carvalho %E Pradeep Ravikumar %F pmlr-v31-wang13c %I PMLR %P 608--617 %U https://proceedings.mlr.press/v31/wang13c.html %V 31 %X The multivariate multi-response (MVMR) linear regression problem is investigated, in which design matrices can be distributed differently across K linear regressions. The support union of K p-dimensional regression vectors are recovered via block regularized Lasso which uses the l_1/l_2 norm for regression vectors across K tasks. Sufficient and necessary conditions to guarantee successful recovery of the support union are characterized. More specifically, it is shown that under certain conditions on the distributions of design matrices, if n > c_p1 ψ(B^*,Σ^(1:K))\log(p-s) where c_p1 is a constant and s is the size of the support set, then the l_1/l_2 regularized Lasso correctly recovers the support union; and if n < c_p2 ψ(B^*,Σ^(1:K))\log(p-s) where c_p2 is a constant, then the l_1/l_2 regularized Lasso fails to recover the support union. In particular, ψ(B^*,Σ^(1:K)) captures the sparsity of K regression vectors and the statistical properties of the design matrices. Numerical results are provided to demonstrate the advantages of joint support union recovery using multi-task Lasso problem over studying each problem individually.
RIS
TY - CPAPER TI - Block Regularized Lasso for Multivariate Multi-Response Linear Regression AU - Weiguang Wang AU - Yingbin Liang AU - Eric Xing BT - Proceedings of the Sixteenth International Conference on Artificial Intelligence and Statistics DA - 2013/04/29 ED - Carlos M. Carvalho ED - Pradeep Ravikumar ID - pmlr-v31-wang13c PB - PMLR DP - Proceedings of Machine Learning Research VL - 31 SP - 608 EP - 617 L1 - http://proceedings.mlr.press/v31/wang13c.pdf UR - https://proceedings.mlr.press/v31/wang13c.html AB - The multivariate multi-response (MVMR) linear regression problem is investigated, in which design matrices can be distributed differently across K linear regressions. The support union of K p-dimensional regression vectors are recovered via block regularized Lasso which uses the l_1/l_2 norm for regression vectors across K tasks. Sufficient and necessary conditions to guarantee successful recovery of the support union are characterized. More specifically, it is shown that under certain conditions on the distributions of design matrices, if n > c_p1 ψ(B^*,Σ^(1:K))\log(p-s) where c_p1 is a constant and s is the size of the support set, then the l_1/l_2 regularized Lasso correctly recovers the support union; and if n < c_p2 ψ(B^*,Σ^(1:K))\log(p-s) where c_p2 is a constant, then the l_1/l_2 regularized Lasso fails to recover the support union. In particular, ψ(B^*,Σ^(1:K)) captures the sparsity of K regression vectors and the statistical properties of the design matrices. Numerical results are provided to demonstrate the advantages of joint support union recovery using multi-task Lasso problem over studying each problem individually. ER -
APA
Wang, W., Liang, Y. & Xing, E.. (2013). Block Regularized Lasso for Multivariate Multi-Response Linear Regression. Proceedings of the Sixteenth International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 31:608-617 Available from https://proceedings.mlr.press/v31/wang13c.html.

Related Material