Improving Adversarial Robustness by Putting More Regularizations on Less Robust Samples

Dongyoon Yang, Insung Kong, Yongdai Kim
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:39331-39348, 2023.

Abstract

Adversarial training, which is to enhance robustness against adversarial attacks, has received much attention because it is easy to generate human-imperceptible perturbations of data to deceive a given deep neural network. In this paper, we propose a new adversarial training algorithm that is theoretically well motivated and empirically superior to other existing algorithms. A novel feature of the proposed algorithm is to apply more regularization to data vulnerable to adversarial attacks than other existing regularization algorithms do. Theoretically, we show that our algorithm can be understood as an algorithm of minimizing a newly derived upper bound of the robust risk. Numerical experiments illustrate that our proposed algorithm improves the generalization (accuracy on examples) and robustness (accuracy on adversarial attacks) simultaneously to achieve the state-of-the-art performance.

Cite this Paper

BibTeX
@InProceedings{pmlr-v202-yang23h, title = {Improving Adversarial Robustness by Putting More Regularizations on Less Robust Samples}, author = {Yang, Dongyoon and Kong, Insung and Kim, Yongdai}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {39331--39348}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/yang23h/yang23h.pdf}, url = {https://proceedings.mlr.press/v202/yang23h.html}, abstract = {Adversarial training, which is to enhance robustness against adversarial attacks, has received much attention because it is easy to generate human-imperceptible perturbations of data to deceive a given deep neural network. In this paper, we propose a new adversarial training algorithm that is theoretically well motivated and empirically superior to other existing algorithms. A novel feature of the proposed algorithm is to apply more regularization to data vulnerable to adversarial attacks than other existing regularization algorithms do. Theoretically, we show that our algorithm can be understood as an algorithm of minimizing a newly derived upper bound of the robust risk. Numerical experiments illustrate that our proposed algorithm improves the generalization (accuracy on examples) and robustness (accuracy on adversarial attacks) simultaneously to achieve the state-of-the-art performance.} }
Endnote
%0 Conference Paper %T Improving Adversarial Robustness by Putting More Regularizations on Less Robust Samples %A Dongyoon Yang %A Insung Kong %A Yongdai Kim %B Proceedings of the 40th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Andreas Krause %E Emma Brunskill %E Kyunghyun Cho %E Barbara Engelhardt %E Sivan Sabato %E Jonathan Scarlett %F pmlr-v202-yang23h %I PMLR %P 39331--39348 %U https://proceedings.mlr.press/v202/yang23h.html %V 202 %X Adversarial training, which is to enhance robustness against adversarial attacks, has received much attention because it is easy to generate human-imperceptible perturbations of data to deceive a given deep neural network. In this paper, we propose a new adversarial training algorithm that is theoretically well motivated and empirically superior to other existing algorithms. A novel feature of the proposed algorithm is to apply more regularization to data vulnerable to adversarial attacks than other existing regularization algorithms do. Theoretically, we show that our algorithm can be understood as an algorithm of minimizing a newly derived upper bound of the robust risk. Numerical experiments illustrate that our proposed algorithm improves the generalization (accuracy on examples) and robustness (accuracy on adversarial attacks) simultaneously to achieve the state-of-the-art performance.
APA
Yang, D., Kong, I. & Kim, Y.. (2023). Improving Adversarial Robustness by Putting More Regularizations on Less Robust Samples. Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:39331-39348 Available from https://proceedings.mlr.press/v202/yang23h.html.

Related Material