Certified Invertibility in Neural Networks via Mixed-Integer Programming

Tianqi Cui, Thomas Bertalan, George J. Pappas, Manfred Morari, Yannis Kevrekidis, Mahyar Fazlyab
Proceedings of The 5th Annual Learning for Dynamics and Control Conference, PMLR 211:483-496, 2023.

Abstract

Neural networks are known to be vulnerable to adversarial attacks, which are small, imperceptible perturbations that can significantly alter the network’s output. Conversely, there may exist large, meaningful perturbations that do not affect the network’s decision (excessive invariance). In our research, we investigate this latter phenomenon in two contexts: (a) discrete-time dynamical system identification, and (b) the calibration of a neural network’s output to that of another network. We examine noninvertibility through the lens of mathematical optimization, where the global solution measures the “safety" of the network predictions by their distance from the non-invertibility boundary. We formulate mixed-integer programs (MIPs) for ReLU networks and $L_p$ norms ($p=1,2,\infty$) that apply to neural network approximators of dynamical systems. We also discuss how our findings can be useful for invertibility certification in transformations between neural networks, e.g. between different levels of network pruning.

Cite this Paper

BibTeX
@InProceedings{pmlr-v211-cui23b, title = {Certified Invertibility in Neural Networks via Mixed-Integer Programming}, author = {Cui, Tianqi and Bertalan, Thomas and Pappas, George J. and Morari, Manfred and Kevrekidis, Yannis and Fazlyab, Mahyar}, booktitle = {Proceedings of The 5th Annual Learning for Dynamics and Control Conference}, pages = {483--496}, year = {2023}, editor = {Matni, Nikolai and Morari, Manfred and Pappas, George J.}, volume = {211}, series = {Proceedings of Machine Learning Research}, month = {15--16 Jun}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v211/cui23b/cui23b.pdf}, url = {https://proceedings.mlr.press/v211/cui23b.html}, abstract = {Neural networks are known to be vulnerable to adversarial attacks, which are small, imperceptible perturbations that can significantly alter the network’s output. Conversely, there may exist large, meaningful perturbations that do not affect the network’s decision (excessive invariance). In our research, we investigate this latter phenomenon in two contexts: (a) discrete-time dynamical system identification, and (b) the calibration of a neural network’s output to that of another network. We examine noninvertibility through the lens of mathematical optimization, where the global solution measures the “safety" of the network predictions by their distance from the non-invertibility boundary. We formulate mixed-integer programs (MIPs) for ReLU networks and $L_p$ norms ($p=1,2,\infty$) that apply to neural network approximators of dynamical systems. We also discuss how our findings can be useful for invertibility certification in transformations between neural networks, e.g. between different levels of network pruning.} }
Endnote
%0 Conference Paper %T Certified Invertibility in Neural Networks via Mixed-Integer Programming %A Tianqi Cui %A Thomas Bertalan %A George J. Pappas %A Manfred Morari %A Yannis Kevrekidis %A Mahyar Fazlyab %B Proceedings of The 5th Annual Learning for Dynamics and Control Conference %C Proceedings of Machine Learning Research %D 2023 %E Nikolai Matni %E Manfred Morari %E George J. Pappas %F pmlr-v211-cui23b %I PMLR %P 483--496 %U https://proceedings.mlr.press/v211/cui23b.html %V 211 %X Neural networks are known to be vulnerable to adversarial attacks, which are small, imperceptible perturbations that can significantly alter the network’s output. Conversely, there may exist large, meaningful perturbations that do not affect the network’s decision (excessive invariance). In our research, we investigate this latter phenomenon in two contexts: (a) discrete-time dynamical system identification, and (b) the calibration of a neural network’s output to that of another network. We examine noninvertibility through the lens of mathematical optimization, where the global solution measures the “safety" of the network predictions by their distance from the non-invertibility boundary. We formulate mixed-integer programs (MIPs) for ReLU networks and $L_p$ norms ($p=1,2,\infty$) that apply to neural network approximators of dynamical systems. We also discuss how our findings can be useful for invertibility certification in transformations between neural networks, e.g. between different levels of network pruning.
APA
Cui, T., Bertalan, T., Pappas, G.J., Morari, M., Kevrekidis, Y. & Fazlyab, M.. (2023). Certified Invertibility in Neural Networks via Mixed-Integer Programming. Proceedings of The 5th Annual Learning for Dynamics and Control Conference, in Proceedings of Machine Learning Research 211:483-496 Available from https://proceedings.mlr.press/v211/cui23b.html.

Related Material