Active Learning of Deep Neural Networks via Gradient-Free Cutting Planes

Erica Zhang, Fangzhao Zhang, Mert Pilanci
Proceedings of the 42nd International Conference on Machine Learning, PMLR 267:76831-76879, 2025.

Abstract

Active learning methods aim to improve sample complexity in machine learning. In this work, we investigate an active learning scheme via a novel gradient-free cutting-plane training method for ReLU networks of arbitrary depth and develop a convergence theory. We demonstrate, for the first time, that cutting-plane algorithms, traditionally used in linear models, can be extended to deep neural networks despite their nonconvexity and nonlinear decision boundaries. Moreover, this training method induces the first deep active learning scheme known to achieve convergence guarantees, revealing a geometric contraction rate of the feasible set. We exemplify the effectiveness of our proposed active learning method against popular deep active learning baselines via both synthetic data experiments and sentimental classification task on real datasets.

Cite this Paper

BibTeX
@InProceedings{pmlr-v267-zhang25db, title = {Active Learning of Deep Neural Networks via Gradient-Free Cutting Planes}, author = {Zhang, Erica and Zhang, Fangzhao and Pilanci, Mert}, booktitle = {Proceedings of the 42nd International Conference on Machine Learning}, pages = {76831--76879}, year = {2025}, editor = {Singh, Aarti and Fazel, Maryam and Hsu, Daniel and Lacoste-Julien, Simon and Berkenkamp, Felix and Maharaj, Tegan and Wagstaff, Kiri and Zhu, Jerry}, volume = {267}, series = {Proceedings of Machine Learning Research}, month = {13--19 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v267/main/assets/zhang25db/zhang25db.pdf}, url = {https://proceedings.mlr.press/v267/zhang25db.html}, abstract = {Active learning methods aim to improve sample complexity in machine learning. In this work, we investigate an active learning scheme via a novel gradient-free cutting-plane training method for ReLU networks of arbitrary depth and develop a convergence theory. We demonstrate, for the first time, that cutting-plane algorithms, traditionally used in linear models, can be extended to deep neural networks despite their nonconvexity and nonlinear decision boundaries. Moreover, this training method induces the first deep active learning scheme known to achieve convergence guarantees, revealing a geometric contraction rate of the feasible set. We exemplify the effectiveness of our proposed active learning method against popular deep active learning baselines via both synthetic data experiments and sentimental classification task on real datasets.} }
Endnote
%0 Conference Paper %T Active Learning of Deep Neural Networks via Gradient-Free Cutting Planes %A Erica Zhang %A Fangzhao Zhang %A Mert Pilanci %B Proceedings of the 42nd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2025 %E Aarti Singh %E Maryam Fazel %E Daniel Hsu %E Simon Lacoste-Julien %E Felix Berkenkamp %E Tegan Maharaj %E Kiri Wagstaff %E Jerry Zhu %F pmlr-v267-zhang25db %I PMLR %P 76831--76879 %U https://proceedings.mlr.press/v267/zhang25db.html %V 267 %X Active learning methods aim to improve sample complexity in machine learning. In this work, we investigate an active learning scheme via a novel gradient-free cutting-plane training method for ReLU networks of arbitrary depth and develop a convergence theory. We demonstrate, for the first time, that cutting-plane algorithms, traditionally used in linear models, can be extended to deep neural networks despite their nonconvexity and nonlinear decision boundaries. Moreover, this training method induces the first deep active learning scheme known to achieve convergence guarantees, revealing a geometric contraction rate of the feasible set. We exemplify the effectiveness of our proposed active learning method against popular deep active learning baselines via both synthetic data experiments and sentimental classification task on real datasets.
APA
Zhang, E., Zhang, F. & Pilanci, M.. (2025). Active Learning of Deep Neural Networks via Gradient-Free Cutting Planes. Proceedings of the 42nd International Conference on Machine Learning, in Proceedings of Machine Learning Research 267:76831-76879 Available from https://proceedings.mlr.press/v267/zhang25db.html.

Related Material