A Dynamical System Perspective for Lipschitz Neural Networks

Laurent Meunier, Blaise J Delattre, Alexandre Araujo, Alexandre Allauzen
Proceedings of the 39th International Conference on Machine Learning, PMLR 162:15484-15500, 2022.

Abstract

The Lipschitz constant of neural networks has been established as a key quantity to enforce the robustness to adversarial examples. In this paper, we tackle the problem of building $1$-Lipschitz Neural Networks. By studying Residual Networks from a continuous time dynamical system perspective, we provide a generic method to build $1$-Lipschitz Neural Networks and show that some previous approaches are special cases of this framework. Then, we extend this reasoning and show that ResNet flows derived from convex potentials define $1$-Lipschitz transformations, that lead us to define the Convex Potential Layer (CPL). A comprehensive set of experiments on several datasets demonstrates the scalability of our architecture and the benefits as an $\ell_2$-provable defense against adversarial examples. Our code is available at \url{https://github.com/MILES-PSL/Convex-Potential-Layer}

Cite this Paper

BibTeX
@InProceedings{pmlr-v162-meunier22a, title = {A Dynamical System Perspective for {L}ipschitz Neural Networks}, author = {Meunier, Laurent and Delattre, Blaise J and Araujo, Alexandre and Allauzen, Alexandre}, booktitle = {Proceedings of the 39th International Conference on Machine Learning}, pages = {15484--15500}, year = {2022}, editor = {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan}, volume = {162}, series = {Proceedings of Machine Learning Research}, month = {17--23 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v162/meunier22a/meunier22a.pdf}, url = {https://proceedings.mlr.press/v162/meunier22a.html}, abstract = {The Lipschitz constant of neural networks has been established as a key quantity to enforce the robustness to adversarial examples. In this paper, we tackle the problem of building $1$-Lipschitz Neural Networks. By studying Residual Networks from a continuous time dynamical system perspective, we provide a generic method to build $1$-Lipschitz Neural Networks and show that some previous approaches are special cases of this framework. Then, we extend this reasoning and show that ResNet flows derived from convex potentials define $1$-Lipschitz transformations, that lead us to define the Convex Potential Layer (CPL). A comprehensive set of experiments on several datasets demonstrates the scalability of our architecture and the benefits as an $\ell_2$-provable defense against adversarial examples. Our code is available at \url{https://github.com/MILES-PSL/Convex-Potential-Layer}} }
Endnote
%0 Conference Paper %T A Dynamical System Perspective for Lipschitz Neural Networks %A Laurent Meunier %A Blaise J Delattre %A Alexandre Araujo %A Alexandre Allauzen %B Proceedings of the 39th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2022 %E Kamalika Chaudhuri %E Stefanie Jegelka %E Le Song %E Csaba Szepesvari %E Gang Niu %E Sivan Sabato %F pmlr-v162-meunier22a %I PMLR %P 15484--15500 %U https://proceedings.mlr.press/v162/meunier22a.html %V 162 %X The Lipschitz constant of neural networks has been established as a key quantity to enforce the robustness to adversarial examples. In this paper, we tackle the problem of building $1$-Lipschitz Neural Networks. By studying Residual Networks from a continuous time dynamical system perspective, we provide a generic method to build $1$-Lipschitz Neural Networks and show that some previous approaches are special cases of this framework. Then, we extend this reasoning and show that ResNet flows derived from convex potentials define $1$-Lipschitz transformations, that lead us to define the Convex Potential Layer (CPL). A comprehensive set of experiments on several datasets demonstrates the scalability of our architecture and the benefits as an $\ell_2$-provable defense against adversarial examples. Our code is available at \url{https://github.com/MILES-PSL/Convex-Potential-Layer}
APA
Meunier, L., Delattre, B.J., Araujo, A. & Allauzen, A.. (2022). A Dynamical System Perspective for Lipschitz Neural Networks. Proceedings of the 39th International Conference on Machine Learning, in Proceedings of Machine Learning Research 162:15484-15500 Available from https://proceedings.mlr.press/v162/meunier22a.html.

Related Material