Sparse Nonlinear Regression: Parameter Estimation under Nonconvexity

Zhuoran Yang; Zhaoran Wang; Han Liu; Yonina Eldar; Tong Zhang

Sparse Nonlinear Regression: Parameter Estimation under Nonconvexity

Zhuoran Yang, Zhaoran Wang, Han Liu, Yonina Eldar, Tong Zhang ;

Proceedings of The 33rd International Conference on Machine Learning, PMLR 48:2472-2481, 2016.

Abstract

We study parameter estimation for sparse nonlinear regression. More specifically, we assume the data are given by y = f( \bf x^T \bf β^* ) + ε, where f is nonlinear. To recover \bf βs, we propose an \ell_1-regularized least-squares estimator. Unlike classical linear regression, the corresponding optimization problem is nonconvex because of the nonlinearity of f. In spite of the nonconvexity, we prove that under mild conditions, every stationary point of the objective enjoys an optimal statistical rate of convergence. Detailed numerical results are provided to back up our theory.

Related Material

@InProceedings{pmlr-v48-yangc16, title = {Sparse Nonlinear Regression: Parameter Estimation under Nonconvexity}, author = {Zhuoran Yang and Zhaoran Wang and Han Liu and Yonina Eldar and Tong Zhang}, booktitle = {Proceedings of The 33rd International Conference on Machine Learning}, pages = {2472--2481}, year = {2016}, editor = {Maria Florina Balcan and Kilian Q. Weinberger}, volume = {48}, series = {Proceedings of Machine Learning Research}, address = {New York, New York, USA}, month = {20--22 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v48/yangc16.pdf}, url = {http://proceedings.mlr.press/v48/yangc16.html}, abstract = {We study parameter estimation for sparse nonlinear regression. More specifically, we assume the data are given by y = f( \bf x^T \bf β^* ) + ε, where f is nonlinear. To recover \bf βs, we propose an \ell_1-regularized least-squares estimator. Unlike classical linear regression, the corresponding optimization problem is nonconvex because of the nonlinearity of f. In spite of the nonconvexity, we prove that under mild conditions, every stationary point of the objective enjoys an optimal statistical rate of convergence. Detailed numerical results are provided to back up our theory.} }

%0 Conference Paper %T Sparse Nonlinear Regression: Parameter Estimation under Nonconvexity %A Zhuoran Yang %A Zhaoran Wang %A Han Liu %A Yonina Eldar %A Tong Zhang %B Proceedings of The 33rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2016 %E Maria Florina Balcan %E Kilian Q. Weinberger %F pmlr-v48-yangc16 %I PMLR %J Proceedings of Machine Learning Research %P 2472--2481 %U http://proceedings.mlr.press %V 48 %W PMLR %X We study parameter estimation for sparse nonlinear regression. More specifically, we assume the data are given by y = f( \bf x^T \bf β^* ) + ε, where f is nonlinear. To recover \bf βs, we propose an \ell_1-regularized least-squares estimator. Unlike classical linear regression, the corresponding optimization problem is nonconvex because of the nonlinearity of f. In spite of the nonconvexity, we prove that under mild conditions, every stationary point of the objective enjoys an optimal statistical rate of convergence. Detailed numerical results are provided to back up our theory.

TY - CPAPER TI - Sparse Nonlinear Regression: Parameter Estimation under Nonconvexity AU - Zhuoran Yang AU - Zhaoran Wang AU - Han Liu AU - Yonina Eldar AU - Tong Zhang BT - Proceedings of The 33rd International Conference on Machine Learning PY - 2016/06/11 DA - 2016/06/11 ED - Maria Florina Balcan ED - Kilian Q. Weinberger ID - pmlr-v48-yangc16 PB - PMLR SP - 2472 DP - PMLR EP - 2481 L1 - http://proceedings.mlr.press/v48/yangc16.pdf UR - http://proceedings.mlr.press/v48/yangc16.html AB - We study parameter estimation for sparse nonlinear regression. More specifically, we assume the data are given by y = f( \bf x^T \bf β^* ) + ε, where f is nonlinear. To recover \bf βs, we propose an \ell_1-regularized least-squares estimator. Unlike classical linear regression, the corresponding optimization problem is nonconvex because of the nonlinearity of f. In spite of the nonconvexity, we prove that under mild conditions, every stationary point of the objective enjoys an optimal statistical rate of convergence. Detailed numerical results are provided to back up our theory. ER -

Yang, Z., Wang, Z., Liu, H., Eldar, Y. & Zhang, T.. (2016). Sparse Nonlinear Regression: Parameter Estimation under Nonconvexity. Proceedings of The 33rd International Conference on Machine Learning, in PMLR 48:2472-2481