Robustly Learning a Single Neuron via Sharpness

Puqian Wang, Nikos Zarifis, Ilias Diakonikolas, Jelena Diakonikolas
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:36541-36577, 2023.

Abstract

We study the problem of learning a single neuron with respect to the $L_2^2$-loss in the presence of adversarial label noise. We give an efficient algorithm that, for a broad family of activations including ReLUs, approximates the optimal $L_2^2$-error within a constant factor. Notably, our algorithm succeeds under much milder distributional assumptions compared to prior work. The key ingredient enabling our results is a novel connection to local error bounds from optimization theory.

Cite this Paper

BibTeX
@InProceedings{pmlr-v202-wang23aq, title = {Robustly Learning a Single Neuron via Sharpness}, author = {Wang, Puqian and Zarifis, Nikos and Diakonikolas, Ilias and Diakonikolas, Jelena}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {36541--36577}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/wang23aq/wang23aq.pdf}, url = {https://proceedings.mlr.press/v202/wang23aq.html}, abstract = {We study the problem of learning a single neuron with respect to the $L_2^2$-loss in the presence of adversarial label noise. We give an efficient algorithm that, for a broad family of activations including ReLUs, approximates the optimal $L_2^2$-error within a constant factor. Notably, our algorithm succeeds under much milder distributional assumptions compared to prior work. The key ingredient enabling our results is a novel connection to local error bounds from optimization theory.} }
Endnote
%0 Conference Paper %T Robustly Learning a Single Neuron via Sharpness %A Puqian Wang %A Nikos Zarifis %A Ilias Diakonikolas %A Jelena Diakonikolas %B Proceedings of the 40th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Andreas Krause %E Emma Brunskill %E Kyunghyun Cho %E Barbara Engelhardt %E Sivan Sabato %E Jonathan Scarlett %F pmlr-v202-wang23aq %I PMLR %P 36541--36577 %U https://proceedings.mlr.press/v202/wang23aq.html %V 202 %X We study the problem of learning a single neuron with respect to the $L_2^2$-loss in the presence of adversarial label noise. We give an efficient algorithm that, for a broad family of activations including ReLUs, approximates the optimal $L_2^2$-error within a constant factor. Notably, our algorithm succeeds under much milder distributional assumptions compared to prior work. The key ingredient enabling our results is a novel connection to local error bounds from optimization theory.
APA
Wang, P., Zarifis, N., Diakonikolas, I. & Diakonikolas, J.. (2023). Robustly Learning a Single Neuron via Sharpness. Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:36541-36577 Available from https://proceedings.mlr.press/v202/wang23aq.html.

Related Material