Few-Sample Feature Selection via Feature Manifold Learning

David Cohen, Tal Shnitzer, Yuval Kluger, Ronen Talmon
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:6296-6319, 2023.

Abstract

In this paper, we present a new method for few-sample supervised feature selection (FS). Our method first learns the manifold of the feature space of each class using kernels capturing multi-feature associations. Then, based on Riemannian geometry, a composite kernel is computed, extracting the differences between the learned feature associations. Finally, a FS score based on spectral analysis is proposed. Considering multi-feature associations makes our method multivariate by design. This in turn allows for the extraction of the hidden manifold underlying the features and avoids overfitting, facilitating few-sample FS. We showcase the efficacy of our method on illustrative examples and several benchmarks, where our method demonstrates higher accuracy in selecting the informative features compared to competing methods. In addition, we show that our FS leads to improved classification and better generalization when applied to test data.

Cite this Paper

BibTeX
@InProceedings{pmlr-v202-cohen23b, title = {Few-Sample Feature Selection via Feature Manifold Learning}, author = {Cohen, David and Shnitzer, Tal and Kluger, Yuval and Talmon, Ronen}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {6296--6319}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/cohen23b/cohen23b.pdf}, url = {https://proceedings.mlr.press/v202/cohen23b.html}, abstract = {In this paper, we present a new method for few-sample supervised feature selection (FS). Our method first learns the manifold of the feature space of each class using kernels capturing multi-feature associations. Then, based on Riemannian geometry, a composite kernel is computed, extracting the differences between the learned feature associations. Finally, a FS score based on spectral analysis is proposed. Considering multi-feature associations makes our method multivariate by design. This in turn allows for the extraction of the hidden manifold underlying the features and avoids overfitting, facilitating few-sample FS. We showcase the efficacy of our method on illustrative examples and several benchmarks, where our method demonstrates higher accuracy in selecting the informative features compared to competing methods. In addition, we show that our FS leads to improved classification and better generalization when applied to test data.} }
Endnote
%0 Conference Paper %T Few-Sample Feature Selection via Feature Manifold Learning %A David Cohen %A Tal Shnitzer %A Yuval Kluger %A Ronen Talmon %B Proceedings of the 40th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Andreas Krause %E Emma Brunskill %E Kyunghyun Cho %E Barbara Engelhardt %E Sivan Sabato %E Jonathan Scarlett %F pmlr-v202-cohen23b %I PMLR %P 6296--6319 %U https://proceedings.mlr.press/v202/cohen23b.html %V 202 %X In this paper, we present a new method for few-sample supervised feature selection (FS). Our method first learns the manifold of the feature space of each class using kernels capturing multi-feature associations. Then, based on Riemannian geometry, a composite kernel is computed, extracting the differences between the learned feature associations. Finally, a FS score based on spectral analysis is proposed. Considering multi-feature associations makes our method multivariate by design. This in turn allows for the extraction of the hidden manifold underlying the features and avoids overfitting, facilitating few-sample FS. We showcase the efficacy of our method on illustrative examples and several benchmarks, where our method demonstrates higher accuracy in selecting the informative features compared to competing methods. In addition, we show that our FS leads to improved classification and better generalization when applied to test data.
APA
Cohen, D., Shnitzer, T., Kluger, Y. & Talmon, R.. (2023). Few-Sample Feature Selection via Feature Manifold Learning. Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:6296-6319 Available from https://proceedings.mlr.press/v202/cohen23b.html.

Related Material