Local Competition and Stochasticity for Adversarial Robustness in Deep Learning

Konstantinos Panousis, Sotirios Chatzis, Antonios Alexos, Sergios Theodoridis
Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, PMLR 130:3862-3870, 2021.

Abstract

This work addresses adversarial robustness in deep learning by considering deep networks with stochastic local winner-takes-all (LWTA) activations. This type of network units result in sparse representations from each model layer, as the units are organized in blocks where only one unit generates a non-zero output. The main operating principle of the introduced units lies on stochastic arguments, as the network performs posterior sampling over competing units to select the winner. We combine these LWTA arguments with tools from the field of Bayesian non-parametrics, specifically the stick-breaking construction of the Indian Buffet Process, to allow for inferring the sub-part of each layer that is essential for modeling the data at hand. Then, inference is performed by means of stochastic variational Bayes. We perform a thorough experimental evaluation of our model using benchmark datasets. As we show, our method achieves high robustness to adversarial perturbations, with state-of-the-art performance in powerful adversarial attack schemes.

Cite this Paper

BibTeX
@InProceedings{pmlr-v130-panousis21a, title = { Local Competition and Stochasticity for Adversarial Robustness in Deep Learning }, author = {Panousis, Konstantinos and Chatzis, Sotirios and Alexos, Antonios and Theodoridis, Sergios}, booktitle = {Proceedings of The 24th International Conference on Artificial Intelligence and Statistics}, pages = {3862--3870}, year = {2021}, editor = {Banerjee, Arindam and Fukumizu, Kenji}, volume = {130}, series = {Proceedings of Machine Learning Research}, month = {13--15 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v130/panousis21a/panousis21a.pdf}, url = {https://proceedings.mlr.press/v130/panousis21a.html}, abstract = { This work addresses adversarial robustness in deep learning by considering deep networks with stochastic local winner-takes-all (LWTA) activations. This type of network units result in sparse representations from each model layer, as the units are organized in blocks where only one unit generates a non-zero output. The main operating principle of the introduced units lies on stochastic arguments, as the network performs posterior sampling over competing units to select the winner. We combine these LWTA arguments with tools from the field of Bayesian non-parametrics, specifically the stick-breaking construction of the Indian Buffet Process, to allow for inferring the sub-part of each layer that is essential for modeling the data at hand. Then, inference is performed by means of stochastic variational Bayes. We perform a thorough experimental evaluation of our model using benchmark datasets. As we show, our method achieves high robustness to adversarial perturbations, with state-of-the-art performance in powerful adversarial attack schemes. } }
Endnote
%0 Conference Paper %T Local Competition and Stochasticity for Adversarial Robustness in Deep Learning %A Konstantinos Panousis %A Sotirios Chatzis %A Antonios Alexos %A Sergios Theodoridis %B Proceedings of The 24th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2021 %E Arindam Banerjee %E Kenji Fukumizu %F pmlr-v130-panousis21a %I PMLR %P 3862--3870 %U https://proceedings.mlr.press/v130/panousis21a.html %V 130 %X This work addresses adversarial robustness in deep learning by considering deep networks with stochastic local winner-takes-all (LWTA) activations. This type of network units result in sparse representations from each model layer, as the units are organized in blocks where only one unit generates a non-zero output. The main operating principle of the introduced units lies on stochastic arguments, as the network performs posterior sampling over competing units to select the winner. We combine these LWTA arguments with tools from the field of Bayesian non-parametrics, specifically the stick-breaking construction of the Indian Buffet Process, to allow for inferring the sub-part of each layer that is essential for modeling the data at hand. Then, inference is performed by means of stochastic variational Bayes. We perform a thorough experimental evaluation of our model using benchmark datasets. As we show, our method achieves high robustness to adversarial perturbations, with state-of-the-art performance in powerful adversarial attack schemes.
APA
Panousis, K., Chatzis, S., Alexos, A. & Theodoridis, S.. (2021). Local Competition and Stochasticity for Adversarial Robustness in Deep Learning . Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 130:3862-3870 Available from https://proceedings.mlr.press/v130/panousis21a.html.

Related Material