Structured nonlinear variable selection

Magda Gregorova, Alexandros Kalousis, Stephane Marchand-Maillet
Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence, PMLR R16:22-31, 2018.

Abstract

We investigate structured sparsity methods for variable selection in regression problems where the target depends nonlinearly on the inputs. We focus on general nonlinear functions not limiting a priori the function space to additive models. We propose two new regularizers based on partial derivatives as nonlinear equivalents of group lasso and elastic net. We formulate the problem within the framework of learning in reproducing kernel Hilbert spaces and show how the variational problem can be reformulated into a more practical finite dimensional equivalent. We develop a new algorithm derived from the ADMM principles that relies solely on closed forms of the proximal operators. We explore the empirical properties of our new algorithm for Nonlinear Variable Selection based on Derivatives (NVSD) on a set of experiments and confirm favourable properties of our structured-sparsity models and the algorithm in terms of both prediction and variable selection accuracy.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR16-gregorova18a, title = {Structured nonlinear variable selection}, author = {Gregorova, Magda and Kalousis, Alexandros and Marchand-Maillet, Stephane}, booktitle = {Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence}, pages = {22--31}, year = {2018}, editor = {Globerson, Amir and Silva, Ricardo}, volume = {R16}, series = {Proceedings of Machine Learning Research}, month = {06--10 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r16/main/assets/gregorova18a/gregorova18a.pdf}, url = {https://proceedings.mlr.press/r16/gregorova18a.html}, abstract = {We investigate structured sparsity methods for variable selection in regression problems where the target depends nonlinearly on the inputs. We focus on general nonlinear functions not limiting a priori the function space to additive models. We propose two new regularizers based on partial derivatives as nonlinear equivalents of group lasso and elastic net. We formulate the problem within the framework of learning in reproducing kernel Hilbert spaces and show how the variational problem can be reformulated into a more practical finite dimensional equivalent. We develop a new algorithm derived from the ADMM principles that relies solely on closed forms of the proximal operators. We explore the empirical properties of our new algorithm for Nonlinear Variable Selection based on Derivatives (NVSD) on a set of experiments and confirm favourable properties of our structured-sparsity models and the algorithm in terms of both prediction and variable selection accuracy.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T Structured nonlinear variable selection %A Magda Gregorova %A Alexandros Kalousis %A Stephane Marchand-Maillet %B Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2018 %E Amir Globerson %E Ricardo Silva %F pmlr-vR16-gregorova18a %I PMLR %P 22--31 %U https://proceedings.mlr.press/r16/gregorova18a.html %V R16 %X We investigate structured sparsity methods for variable selection in regression problems where the target depends nonlinearly on the inputs. We focus on general nonlinear functions not limiting a priori the function space to additive models. We propose two new regularizers based on partial derivatives as nonlinear equivalents of group lasso and elastic net. We formulate the problem within the framework of learning in reproducing kernel Hilbert spaces and show how the variational problem can be reformulated into a more practical finite dimensional equivalent. We develop a new algorithm derived from the ADMM principles that relies solely on closed forms of the proximal operators. We explore the empirical properties of our new algorithm for Nonlinear Variable Selection based on Derivatives (NVSD) on a set of experiments and confirm favourable properties of our structured-sparsity models and the algorithm in terms of both prediction and variable selection accuracy. %Z Reissued by PMLR on 04 October 2026.
APA
Gregorova, M., Kalousis, A. & Marchand-Maillet, S.. (2018). Structured nonlinear variable selection. Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R16:22-31 Available from https://proceedings.mlr.press/r16/gregorova18a.html. Reissued by PMLR on 04 October 2026.

Related Material