Weight-covariance alignment for adversarially robust neural networks

Panagiotis Eustratiadis, Henry Gouk, Da Li, Timothy Hospedales
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:3047-3056, 2021.

Abstract

Stochastic Neural Networks (SNNs) that inject noise into their hidden layers have recently been shown to achieve strong robustness against adversarial attacks. However, existing SNNs are usually heuristically motivated, and often rely on adversarial training, which is computationally costly. We propose a new SNN that achieves state-of-the-art performance without relying on adversarial training, and enjoys solid theoretical justification. Specifically, while existing SNNs inject learned or hand-tuned isotropic noise, our SNN learns an anisotropic noise distribution to optimize a learning-theoretic bound on adversarial robustness. We evaluate our method on a number of popular benchmarks, show that it can be applied to different architectures, and that it provides robustness to a variety of white-box and black-box attacks, while being simple and fast to train compared to existing alternatives.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-eustratiadis21a, title = {Weight-covariance alignment for adversarially robust neural networks}, author = {Eustratiadis, Panagiotis and Gouk, Henry and Li, Da and Hospedales, Timothy}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {3047--3056}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/eustratiadis21a/eustratiadis21a.pdf}, url = {https://proceedings.mlr.press/v139/eustratiadis21a.html}, abstract = {Stochastic Neural Networks (SNNs) that inject noise into their hidden layers have recently been shown to achieve strong robustness against adversarial attacks. However, existing SNNs are usually heuristically motivated, and often rely on adversarial training, which is computationally costly. We propose a new SNN that achieves state-of-the-art performance without relying on adversarial training, and enjoys solid theoretical justification. Specifically, while existing SNNs inject learned or hand-tuned isotropic noise, our SNN learns an anisotropic noise distribution to optimize a learning-theoretic bound on adversarial robustness. We evaluate our method on a number of popular benchmarks, show that it can be applied to different architectures, and that it provides robustness to a variety of white-box and black-box attacks, while being simple and fast to train compared to existing alternatives.} }
Endnote
%0 Conference Paper %T Weight-covariance alignment for adversarially robust neural networks %A Panagiotis Eustratiadis %A Henry Gouk %A Da Li %A Timothy Hospedales %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-eustratiadis21a %I PMLR %P 3047--3056 %U https://proceedings.mlr.press/v139/eustratiadis21a.html %V 139 %X Stochastic Neural Networks (SNNs) that inject noise into their hidden layers have recently been shown to achieve strong robustness against adversarial attacks. However, existing SNNs are usually heuristically motivated, and often rely on adversarial training, which is computationally costly. We propose a new SNN that achieves state-of-the-art performance without relying on adversarial training, and enjoys solid theoretical justification. Specifically, while existing SNNs inject learned or hand-tuned isotropic noise, our SNN learns an anisotropic noise distribution to optimize a learning-theoretic bound on adversarial robustness. We evaluate our method on a number of popular benchmarks, show that it can be applied to different architectures, and that it provides robustness to a variety of white-box and black-box attacks, while being simple and fast to train compared to existing alternatives.
APA
Eustratiadis, P., Gouk, H., Li, D. & Hospedales, T.. (2021). Weight-covariance alignment for adversarially robust neural networks. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:3047-3056 Available from https://proceedings.mlr.press/v139/eustratiadis21a.html.

Related Material