Sparse Nonlinear Regression: Parameter Estimation under Nonconvexity


Zhuoran Yang, Zhaoran Wang, Han Liu, Yonina Eldar, Tong Zhang ;
Proceedings of The 33rd International Conference on Machine Learning, PMLR 48:2472-2481, 2016.


We study parameter estimation for sparse nonlinear regression. More specifically, we assume the data are given by y = f( \bf x^T \bf β^* ) + ε, where f is nonlinear. To recover \bf βs, we propose an \ell_1-regularized least-squares estimator. Unlike classical linear regression, the corresponding optimization problem is nonconvex because of the nonlinearity of f. In spite of the nonconvexity, we prove that under mild conditions, every stationary point of the objective enjoys an optimal statistical rate of convergence. Detailed numerical results are provided to back up our theory.

Related Material