Fundamental Tradeoffs between Invariance and Sensitivity to Adversarial Perturbations

Florian Tramer, Jens Behrmann, Nicholas Carlini, Nicolas Papernot, Joern-Henrik Jacobsen
Proceedings of the 37th International Conference on Machine Learning, PMLR 119:9561-9571, 2020.

Abstract

Adversarial examples are malicious inputs crafted to induce misclassification. Commonly studied \emph{sensitivity-based} adversarial examples introduce semantically-small changes to an input that result in a different model prediction. This paper studies a complementary failure mode, \emph{invariance-based} adversarial examples, that introduce minimal semantic changes that modify an input’s true label yet preserve the model’s prediction. We demonstrate fundamental tradeoffs between these two types of adversarial examples. We show that defenses against sensitivity-based attacks actively harm a model’s accuracy on invariance-based attacks, and that new approaches are needed to resist both attack types. In particular, we break state-of-the-art adversarially-trained and \emph{certifiably-robust} models by generating small perturbations that the models are (provably) robust to, yet that change an input’s class according to human labelers. Finally, we formally show that the existence of excessively invariant classifiers arises from the presence of \emph{overly-robust} predictive features in standard datasets.

Cite this Paper

BibTeX
@InProceedings{pmlr-v119-tramer20a, title = {Fundamental Tradeoffs between Invariance and Sensitivity to Adversarial Perturbations}, author = {Tramer, Florian and Behrmann, Jens and Carlini, Nicholas and Papernot, Nicolas and Jacobsen, Joern-Henrik}, booktitle = {Proceedings of the 37th International Conference on Machine Learning}, pages = {9561--9571}, year = {2020}, editor = {III, Hal Daumé and Singh, Aarti}, volume = {119}, series = {Proceedings of Machine Learning Research}, month = {13--18 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v119/tramer20a/tramer20a.pdf}, url = {https://proceedings.mlr.press/v119/tramer20a.html}, abstract = {Adversarial examples are malicious inputs crafted to induce misclassification. Commonly studied \emph{sensitivity-based} adversarial examples introduce semantically-small changes to an input that result in a different model prediction. This paper studies a complementary failure mode, \emph{invariance-based} adversarial examples, that introduce minimal semantic changes that modify an input’s true label yet preserve the model’s prediction. We demonstrate fundamental tradeoffs between these two types of adversarial examples. We show that defenses against sensitivity-based attacks actively harm a model’s accuracy on invariance-based attacks, and that new approaches are needed to resist both attack types. In particular, we break state-of-the-art adversarially-trained and \emph{certifiably-robust} models by generating small perturbations that the models are (provably) robust to, yet that change an input’s class according to human labelers. Finally, we formally show that the existence of excessively invariant classifiers arises from the presence of \emph{overly-robust} predictive features in standard datasets.} }
Endnote
%0 Conference Paper %T Fundamental Tradeoffs between Invariance and Sensitivity to Adversarial Perturbations %A Florian Tramer %A Jens Behrmann %A Nicholas Carlini %A Nicolas Papernot %A Joern-Henrik Jacobsen %B Proceedings of the 37th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2020 %E Hal Daumé III %E Aarti Singh %F pmlr-v119-tramer20a %I PMLR %P 9561--9571 %U https://proceedings.mlr.press/v119/tramer20a.html %V 119 %X Adversarial examples are malicious inputs crafted to induce misclassification. Commonly studied \emph{sensitivity-based} adversarial examples introduce semantically-small changes to an input that result in a different model prediction. This paper studies a complementary failure mode, \emph{invariance-based} adversarial examples, that introduce minimal semantic changes that modify an input’s true label yet preserve the model’s prediction. We demonstrate fundamental tradeoffs between these two types of adversarial examples. We show that defenses against sensitivity-based attacks actively harm a model’s accuracy on invariance-based attacks, and that new approaches are needed to resist both attack types. In particular, we break state-of-the-art adversarially-trained and \emph{certifiably-robust} models by generating small perturbations that the models are (provably) robust to, yet that change an input’s class according to human labelers. Finally, we formally show that the existence of excessively invariant classifiers arises from the presence of \emph{overly-robust} predictive features in standard datasets.
APA
Tramer, F., Behrmann, J., Carlini, N., Papernot, N. & Jacobsen, J.. (2020). Fundamental Tradeoffs between Invariance and Sensitivity to Adversarial Perturbations. Proceedings of the 37th International Conference on Machine Learning, in Proceedings of Machine Learning Research 119:9561-9571 Available from https://proceedings.mlr.press/v119/tramer20a.html.

Related Material