Effective Nonlinear Feature Selection Method based on HSIC Lasso and with Variational Inference

Kazuki Koyama, Keisuke Kiritoshi, Tomomi Okawachi, Tomonori Izumitani
Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, PMLR 151:10407-10421, 2022.

Abstract

HSIC Lasso is one of the most effective sparse nonlinear feature selection methods based on the Hilbert-Schmidt independence criterion. We propose an adaptive nonlinear feature selection method, which is based on the HSIC Lasso, that uses a stochastic model with a family of super-Gaussian prior distributions for sparsity enhancement. The method includes easily implementable closed-form update equations that are derived approximately from variational inference and can handle high-dimensional and large datasets. We applied the method to several synthetic datasets and real-world datasets and verified its effectiveness regarding redundancy, computational complexity, and classification and prediction accuracy using the selected features. The results indicate that the method can more effectively remove irrelevant features, leaving only relevant features. In certain problem settings, the method assigned non-zero importance only to the actually relevant features. This is an important characteristic for practical use.

Cite this Paper

BibTeX
@InProceedings{pmlr-v151-koyama22a, title = { Effective Nonlinear Feature Selection Method based on HSIC Lasso and with Variational Inference }, author = {Koyama, Kazuki and Kiritoshi, Keisuke and Okawachi, Tomomi and Izumitani, Tomonori}, booktitle = {Proceedings of The 25th International Conference on Artificial Intelligence and Statistics}, pages = {10407--10421}, year = {2022}, editor = {Camps-Valls, Gustau and Ruiz, Francisco J. R. and Valera, Isabel}, volume = {151}, series = {Proceedings of Machine Learning Research}, month = {28--30 Mar}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v151/koyama22a/koyama22a.pdf}, url = {https://proceedings.mlr.press/v151/koyama22a.html}, abstract = { HSIC Lasso is one of the most effective sparse nonlinear feature selection methods based on the Hilbert-Schmidt independence criterion. We propose an adaptive nonlinear feature selection method, which is based on the HSIC Lasso, that uses a stochastic model with a family of super-Gaussian prior distributions for sparsity enhancement. The method includes easily implementable closed-form update equations that are derived approximately from variational inference and can handle high-dimensional and large datasets. We applied the method to several synthetic datasets and real-world datasets and verified its effectiveness regarding redundancy, computational complexity, and classification and prediction accuracy using the selected features. The results indicate that the method can more effectively remove irrelevant features, leaving only relevant features. In certain problem settings, the method assigned non-zero importance only to the actually relevant features. This is an important characteristic for practical use. } }
Endnote
%0 Conference Paper %T Effective Nonlinear Feature Selection Method based on HSIC Lasso and with Variational Inference %A Kazuki Koyama %A Keisuke Kiritoshi %A Tomomi Okawachi %A Tomonori Izumitani %B Proceedings of The 25th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2022 %E Gustau Camps-Valls %E Francisco J. R. Ruiz %E Isabel Valera %F pmlr-v151-koyama22a %I PMLR %P 10407--10421 %U https://proceedings.mlr.press/v151/koyama22a.html %V 151 %X HSIC Lasso is one of the most effective sparse nonlinear feature selection methods based on the Hilbert-Schmidt independence criterion. We propose an adaptive nonlinear feature selection method, which is based on the HSIC Lasso, that uses a stochastic model with a family of super-Gaussian prior distributions for sparsity enhancement. The method includes easily implementable closed-form update equations that are derived approximately from variational inference and can handle high-dimensional and large datasets. We applied the method to several synthetic datasets and real-world datasets and verified its effectiveness regarding redundancy, computational complexity, and classification and prediction accuracy using the selected features. The results indicate that the method can more effectively remove irrelevant features, leaving only relevant features. In certain problem settings, the method assigned non-zero importance only to the actually relevant features. This is an important characteristic for practical use.
APA
Koyama, K., Kiritoshi, K., Okawachi, T. & Izumitani, T.. (2022). Effective Nonlinear Feature Selection Method based on HSIC Lasso and with Variational Inference . Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 151:10407-10421 Available from https://proceedings.mlr.press/v151/koyama22a.html.

Related Material