A New Branch-and-Bound Pruning Framework for $\ell_0$-Regularized Problems

Guyard Theo, Cédric Herzet, Clément Elvira, Ayse-Nur Arslan
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:48077-48096, 2024.

Abstract

We consider the resolution of learning problems involving $\ell_0$-regularization via Branch-and- Bound (BnB) algorithms. These methods explore regions of the feasible space of the problem and check whether they do not contain solutions through “pruning tests”. In standard implementations, evaluating a pruning test requires to solve a convex optimization problem, which may result in computational bottlenecks. In this paper, we present an alternative to implement pruning tests for some generic family of $\ell_0$-regularized problems. Our proposed procedure allows the simultaneous assessment of several regions and can be embedded in standard BnB implementations with a negligible computational overhead. We show through numerical simulations that our pruning strategy can improve the solving time of BnB procedures by several orders of magnitude for typical problems encountered in machine-learning applications.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-theo24a, title = {A New Branch-and-Bound Pruning Framework for $\ell_0$-Regularized Problems}, author = {Theo, Guyard and Herzet, C\'{e}dric and Elvira, Cl\'{e}ment and Arslan, Ayse-Nur}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {48077--48096}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/theo24a/theo24a.pdf}, url = {https://proceedings.mlr.press/v235/theo24a.html}, abstract = {We consider the resolution of learning problems involving $\ell_0$-regularization via Branch-and- Bound (BnB) algorithms. These methods explore regions of the feasible space of the problem and check whether they do not contain solutions through “pruning tests”. In standard implementations, evaluating a pruning test requires to solve a convex optimization problem, which may result in computational bottlenecks. In this paper, we present an alternative to implement pruning tests for some generic family of $\ell_0$-regularized problems. Our proposed procedure allows the simultaneous assessment of several regions and can be embedded in standard BnB implementations with a negligible computational overhead. We show through numerical simulations that our pruning strategy can improve the solving time of BnB procedures by several orders of magnitude for typical problems encountered in machine-learning applications.} }
Endnote
%0 Conference Paper %T A New Branch-and-Bound Pruning Framework for $\ell_0$-Regularized Problems %A Guyard Theo %A Cédric Herzet %A Clément Elvira %A Ayse-Nur Arslan %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-theo24a %I PMLR %P 48077--48096 %U https://proceedings.mlr.press/v235/theo24a.html %V 235 %X We consider the resolution of learning problems involving $\ell_0$-regularization via Branch-and- Bound (BnB) algorithms. These methods explore regions of the feasible space of the problem and check whether they do not contain solutions through “pruning tests”. In standard implementations, evaluating a pruning test requires to solve a convex optimization problem, which may result in computational bottlenecks. In this paper, we present an alternative to implement pruning tests for some generic family of $\ell_0$-regularized problems. Our proposed procedure allows the simultaneous assessment of several regions and can be embedded in standard BnB implementations with a negligible computational overhead. We show through numerical simulations that our pruning strategy can improve the solving time of BnB procedures by several orders of magnitude for typical problems encountered in machine-learning applications.
APA
Theo, G., Herzet, C., Elvira, C. & Arslan, A.. (2024). A New Branch-and-Bound Pruning Framework for $\ell_0$-Regularized Problems. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:48077-48096 Available from https://proceedings.mlr.press/v235/theo24a.html.

Related Material