Neural Network Control Policy Verification With Persistent Adversarial Perturbation

Yuh-Shyang Wang, Lily Weng, Luca Daniel
Proceedings of the 37th International Conference on Machine Learning, PMLR 119:10050-10059, 2020.

Abstract

Deep neural networks are known to be fragile to small adversarial perturbations, which raises serious concerns when a neural network policy is interconnected with a physical system in a closed loop. In this paper, we show how to combine recent works on static neural network certification tools with robust control theory to certify a neural network policy in a control loop. We give a sufficient condition and an algorithm to ensure that the closed loop state and control constraints are satisfied when the persistent adversarial perturbation is l-infinity norm bounded. Our method is based on finding a positively invariant set of the closed loop dynamical system, and thus we do not require the continuity of the neural network policy. Along with the verification result, we also develop an effective attack strategy for neural network control systems that outperforms exhaustive Monte-Carlo search significantly. We show that our certification algorithm works well on learned models and could achieve 5 times better result than the traditional Lipschitz-based method to certify the robustness of a neural network policy on the cart-pole balance control problem.

Cite this Paper

BibTeX
@InProceedings{pmlr-v119-wang20v, title = {Neural Network Control Policy Verification With Persistent Adversarial Perturbation}, author = {Wang, Yuh-Shyang and Weng, Lily and Daniel, Luca}, booktitle = {Proceedings of the 37th International Conference on Machine Learning}, pages = {10050--10059}, year = {2020}, editor = {III, Hal Daumé and Singh, Aarti}, volume = {119}, series = {Proceedings of Machine Learning Research}, month = {13--18 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v119/wang20v/wang20v.pdf}, url = {https://proceedings.mlr.press/v119/wang20v.html}, abstract = {Deep neural networks are known to be fragile to small adversarial perturbations, which raises serious concerns when a neural network policy is interconnected with a physical system in a closed loop. In this paper, we show how to combine recent works on static neural network certification tools with robust control theory to certify a neural network policy in a control loop. We give a sufficient condition and an algorithm to ensure that the closed loop state and control constraints are satisfied when the persistent adversarial perturbation is l-infinity norm bounded. Our method is based on finding a positively invariant set of the closed loop dynamical system, and thus we do not require the continuity of the neural network policy. Along with the verification result, we also develop an effective attack strategy for neural network control systems that outperforms exhaustive Monte-Carlo search significantly. We show that our certification algorithm works well on learned models and could achieve 5 times better result than the traditional Lipschitz-based method to certify the robustness of a neural network policy on the cart-pole balance control problem.} }
Endnote
%0 Conference Paper %T Neural Network Control Policy Verification With Persistent Adversarial Perturbation %A Yuh-Shyang Wang %A Lily Weng %A Luca Daniel %B Proceedings of the 37th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2020 %E Hal Daumé III %E Aarti Singh %F pmlr-v119-wang20v %I PMLR %P 10050--10059 %U https://proceedings.mlr.press/v119/wang20v.html %V 119 %X Deep neural networks are known to be fragile to small adversarial perturbations, which raises serious concerns when a neural network policy is interconnected with a physical system in a closed loop. In this paper, we show how to combine recent works on static neural network certification tools with robust control theory to certify a neural network policy in a control loop. We give a sufficient condition and an algorithm to ensure that the closed loop state and control constraints are satisfied when the persistent adversarial perturbation is l-infinity norm bounded. Our method is based on finding a positively invariant set of the closed loop dynamical system, and thus we do not require the continuity of the neural network policy. Along with the verification result, we also develop an effective attack strategy for neural network control systems that outperforms exhaustive Monte-Carlo search significantly. We show that our certification algorithm works well on learned models and could achieve 5 times better result than the traditional Lipschitz-based method to certify the robustness of a neural network policy on the cart-pole balance control problem.
APA
Wang, Y., Weng, L. & Daniel, L.. (2020). Neural Network Control Policy Verification With Persistent Adversarial Perturbation. Proceedings of the 37th International Conference on Machine Learning, in Proceedings of Machine Learning Research 119:10050-10059 Available from https://proceedings.mlr.press/v119/wang20v.html.

Related Material