A Unified Perspective on Regularization and Perturbation in Differentiable Subset Selection

Xiangqian Sun, Cheuk Hang Leung, Yijun Li, Qi Wu
Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, PMLR 206:4629-4642, 2023.

Abstract

Subset selection, i.e., finding a bunch of items from a collection to achieve specific goals, has wide applications in information retrieval, statistics, and machine learning. To implement an end-to-end learning framework, different relaxed differentiable operators of subset selection are proposed. Most existing work relies on either the regularization method or the perturbation method. In this work, we provide a probabilistic interpretation for regularization relaxation and unify two schemes. Besides, we build some concrete examples to show the generic connection between these two relaxations. Finally, we evaluate the perturbed selector as well as the regularized selector on two tasks: the maximum entropy sampling problem and the feature selection problem. The experimental results show that these two methods can achieve competitive performance against other benchmarks.

Cite this Paper

BibTeX
@InProceedings{pmlr-v206-sun23e, title = {A Unified Perspective on Regularization and Perturbation in Differentiable Subset Selection}, author = {Sun, Xiangqian and Leung, Cheuk Hang and Li, Yijun and Wu, Qi}, booktitle = {Proceedings of The 26th International Conference on Artificial Intelligence and Statistics}, pages = {4629--4642}, year = {2023}, editor = {Ruiz, Francisco and Dy, Jennifer and van de Meent, Jan-Willem}, volume = {206}, series = {Proceedings of Machine Learning Research}, month = {25--27 Apr}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v206/sun23e/sun23e.pdf}, url = {https://proceedings.mlr.press/v206/sun23e.html}, abstract = {Subset selection, i.e., finding a bunch of items from a collection to achieve specific goals, has wide applications in information retrieval, statistics, and machine learning. To implement an end-to-end learning framework, different relaxed differentiable operators of subset selection are proposed. Most existing work relies on either the regularization method or the perturbation method. In this work, we provide a probabilistic interpretation for regularization relaxation and unify two schemes. Besides, we build some concrete examples to show the generic connection between these two relaxations. Finally, we evaluate the perturbed selector as well as the regularized selector on two tasks: the maximum entropy sampling problem and the feature selection problem. The experimental results show that these two methods can achieve competitive performance against other benchmarks.} }
Endnote
%0 Conference Paper %T A Unified Perspective on Regularization and Perturbation in Differentiable Subset Selection %A Xiangqian Sun %A Cheuk Hang Leung %A Yijun Li %A Qi Wu %B Proceedings of The 26th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2023 %E Francisco Ruiz %E Jennifer Dy %E Jan-Willem van de Meent %F pmlr-v206-sun23e %I PMLR %P 4629--4642 %U https://proceedings.mlr.press/v206/sun23e.html %V 206 %X Subset selection, i.e., finding a bunch of items from a collection to achieve specific goals, has wide applications in information retrieval, statistics, and machine learning. To implement an end-to-end learning framework, different relaxed differentiable operators of subset selection are proposed. Most existing work relies on either the regularization method or the perturbation method. In this work, we provide a probabilistic interpretation for regularization relaxation and unify two schemes. Besides, we build some concrete examples to show the generic connection between these two relaxations. Finally, we evaluate the perturbed selector as well as the regularized selector on two tasks: the maximum entropy sampling problem and the feature selection problem. The experimental results show that these two methods can achieve competitive performance against other benchmarks.
APA
Sun, X., Leung, C.H., Li, Y. & Wu, Q.. (2023). A Unified Perspective on Regularization and Perturbation in Differentiable Subset Selection. Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 206:4629-4642 Available from https://proceedings.mlr.press/v206/sun23e.html.

Related Material