Towards a Theoretical Understanding of the Robustness of Variational Autoencoders

Alexander Camuto, Matthew Willetts, Stephen Roberts, Chris Holmes, Tom Rainforth
Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, PMLR 130:3565-3573, 2021.

Abstract

We make inroads into understanding the robustness of Variational Autoencoders (VAEs) to adversarial attacks and other input perturbations. While previous work has developed algorithmic approaches to attacking and defending VAEs, there remains a lack of formalization for what it means for a VAE to be robust. To address this, we develop a novel criterion for robustness in probabilistic models: $r$-robustness. We then use this to construct the first theoretical results for the robustness of VAEs, deriving margins in the input space for which we can provide guarantees about the resulting reconstruction. Informally, we are able to define a region within which any perturbation will produce a reconstruction that is similar to the original reconstruction. To support our analysis, we show that VAEs trained using disentangling methods not only score well under our robustness metrics, but that the reasons for this can be interpreted through our theoretical results.

Cite this Paper

BibTeX
@InProceedings{pmlr-v130-camuto21a, title = { Towards a Theoretical Understanding of the Robustness of Variational Autoencoders }, author = {Camuto, Alexander and Willetts, Matthew and Roberts, Stephen and Holmes, Chris and Rainforth, Tom}, booktitle = {Proceedings of The 24th International Conference on Artificial Intelligence and Statistics}, pages = {3565--3573}, year = {2021}, editor = {Banerjee, Arindam and Fukumizu, Kenji}, volume = {130}, series = {Proceedings of Machine Learning Research}, month = {13--15 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v130/camuto21a/camuto21a.pdf}, url = {https://proceedings.mlr.press/v130/camuto21a.html}, abstract = { We make inroads into understanding the robustness of Variational Autoencoders (VAEs) to adversarial attacks and other input perturbations. While previous work has developed algorithmic approaches to attacking and defending VAEs, there remains a lack of formalization for what it means for a VAE to be robust. To address this, we develop a novel criterion for robustness in probabilistic models: $r$-robustness. We then use this to construct the first theoretical results for the robustness of VAEs, deriving margins in the input space for which we can provide guarantees about the resulting reconstruction. Informally, we are able to define a region within which any perturbation will produce a reconstruction that is similar to the original reconstruction. To support our analysis, we show that VAEs trained using disentangling methods not only score well under our robustness metrics, but that the reasons for this can be interpreted through our theoretical results. } }
Endnote
%0 Conference Paper %T Towards a Theoretical Understanding of the Robustness of Variational Autoencoders %A Alexander Camuto %A Matthew Willetts %A Stephen Roberts %A Chris Holmes %A Tom Rainforth %B Proceedings of The 24th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2021 %E Arindam Banerjee %E Kenji Fukumizu %F pmlr-v130-camuto21a %I PMLR %P 3565--3573 %U https://proceedings.mlr.press/v130/camuto21a.html %V 130 %X We make inroads into understanding the robustness of Variational Autoencoders (VAEs) to adversarial attacks and other input perturbations. While previous work has developed algorithmic approaches to attacking and defending VAEs, there remains a lack of formalization for what it means for a VAE to be robust. To address this, we develop a novel criterion for robustness in probabilistic models: $r$-robustness. We then use this to construct the first theoretical results for the robustness of VAEs, deriving margins in the input space for which we can provide guarantees about the resulting reconstruction. Informally, we are able to define a region within which any perturbation will produce a reconstruction that is similar to the original reconstruction. To support our analysis, we show that VAEs trained using disentangling methods not only score well under our robustness metrics, but that the reasons for this can be interpreted through our theoretical results.
APA
Camuto, A., Willetts, M., Roberts, S., Holmes, C. & Rainforth, T.. (2021). Towards a Theoretical Understanding of the Robustness of Variational Autoencoders . Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 130:3565-3573 Available from https://proceedings.mlr.press/v130/camuto21a.html.

Related Material