Structured nonlinear variable selection

Magda Gregorova, Alexandros Kalousis, Stephane Marchand-Maillet
Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence, PMLR R16:22-31, 2018.

Abstract

We investigate structured sparsity methods for variable selection in regression problems where the target depends nonlinearly on the inputs. We focus on general nonlinear func- tions not limiting a priori the function space to additive models. We propose two new regu- larizers based on partial derivatives as nonlin- ear equivalents of group lasso and elastic net. We formulate the problem within the frame- work of learning in reproducing kernel Hilbert spaces and show how the variational problem can be reformulated into a more practical fi- nite dimensional equivalent. We develop a new algorithm derived from the ADMM principles that relies solely on closed forms of the proxi- mal operators. We explore the empirical prop- erties of our new algorithm for Nonlinear Vari- able Selection based on Derivatives (NVSD) on a set of experiments and confirm favourable properties of our structured-sparsity models and the algorithm in terms of both prediction and variable selection accuracy.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR16-gregorova18a, title = {Structured nonlinear variable selection}, author = {Gregorova, Magda and Kalousis, Alexandros and Marchand-Maillet, Stephane}, booktitle = {Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence}, pages = {22--31}, year = {2018}, editor = {Globerson, Amir and Silva, Ricardo}, volume = {R16}, series = {Proceedings of Machine Learning Research}, month = {06--10 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r16/main/assets/gregorova18a/gregorova18a.pdf}, url = {https://proceedings.mlr.press/r16/gregorova18a.html}, abstract = {We investigate structured sparsity methods for variable selection in regression problems where the target depends nonlinearly on the inputs. We focus on general nonlinear func- tions not limiting a priori the function space to additive models. We propose two new regu- larizers based on partial derivatives as nonlin- ear equivalents of group lasso and elastic net. We formulate the problem within the frame- work of learning in reproducing kernel Hilbert spaces and show how the variational problem can be reformulated into a more practical fi- nite dimensional equivalent. We develop a new algorithm derived from the ADMM principles that relies solely on closed forms of the proxi- mal operators. We explore the empirical prop- erties of our new algorithm for Nonlinear Vari- able Selection based on Derivatives (NVSD) on a set of experiments and confirm favourable properties of our structured-sparsity models and the algorithm in terms of both prediction and variable selection accuracy.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T Structured nonlinear variable selection %A Magda Gregorova %A Alexandros Kalousis %A Stephane Marchand-Maillet %B Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2018 %E Amir Globerson %E Ricardo Silva %F pmlr-vR16-gregorova18a %I PMLR %P 22--31 %U https://proceedings.mlr.press/r16/gregorova18a.html %V R16 %X We investigate structured sparsity methods for variable selection in regression problems where the target depends nonlinearly on the inputs. We focus on general nonlinear func- tions not limiting a priori the function space to additive models. We propose two new regu- larizers based on partial derivatives as nonlin- ear equivalents of group lasso and elastic net. We formulate the problem within the frame- work of learning in reproducing kernel Hilbert spaces and show how the variational problem can be reformulated into a more practical fi- nite dimensional equivalent. We develop a new algorithm derived from the ADMM principles that relies solely on closed forms of the proxi- mal operators. We explore the empirical prop- erties of our new algorithm for Nonlinear Vari- able Selection based on Derivatives (NVSD) on a set of experiments and confirm favourable properties of our structured-sparsity models and the algorithm in terms of both prediction and variable selection accuracy. %Z Reissued by PMLR on 04 October 2026.
APA
Gregorova, M., Kalousis, A. & Marchand-Maillet, S.. (2018). Structured nonlinear variable selection. Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R16:22-31 Available from https://proceedings.mlr.press/r16/gregorova18a.html. Reissued by PMLR on 04 October 2026.

Related Material