Outlier Robust Adversarial Training

Shu Hu, Zhenhuan Yang, Xin Wang, Yiming Ying, Siwei Lyu
Proceedings of the 15th Asian Conference on Machine Learning, PMLR 222:454-469, 2024.

Abstract

Supervised learning models are challenged by the intrinsic complexities of training data such as outliers and minority subpopulations and intentional attacks at inference time with adversarial samples. While traditional robust learning methods and the recent adversarial training approaches are designed to handle each of the two challenges, to date, no work has been done to develop models that are robust with regard to the low-quality training data and the potential adversarial attack at inference time simultaneously. It is for this reason that we introduce \underline{O}utlier \underline{R}obust \underline{A}dversarial \underline{T}raining (ORAT) in this work. ORAT is based on a bi-level optimization formulation of adversarial training with a robust rank-based loss function. Theoretically, we show that the learning objective of ORAT satisfies the $\mathcal{H}$-consistency in binary classification, which establishes it as a proper surrogate to adversarial 0/1 loss. Furthermore, we analyze its generalization ability and provide uniform convergence rates in high probability. ORAT can be optimized with a simple algorithm. Experimental evaluations on three benchmark datasets demonstrate the effectiveness and robustness of ORAT in handling outliers and adversarial attacks. Our code is available at \url{https://github.com/discovershu/ORAT}.

Cite this Paper

BibTeX
@InProceedings{pmlr-v222-hu24a, title = {Outlier Robust Adversarial Training}, author = {Hu, Shu and Yang, Zhenhuan and Wang, Xin and Ying, Yiming and Lyu, Siwei}, booktitle = {Proceedings of the 15th Asian Conference on Machine Learning}, pages = {454--469}, year = {2024}, editor = {Yanıkoğlu, Berrin and Buntine, Wray}, volume = {222}, series = {Proceedings of Machine Learning Research}, month = {11--14 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v222/hu24a/hu24a.pdf}, url = {https://proceedings.mlr.press/v222/hu24a.html}, abstract = {Supervised learning models are challenged by the intrinsic complexities of training data such as outliers and minority subpopulations and intentional attacks at inference time with adversarial samples. While traditional robust learning methods and the recent adversarial training approaches are designed to handle each of the two challenges, to date, no work has been done to develop models that are robust with regard to the low-quality training data and the potential adversarial attack at inference time simultaneously. It is for this reason that we introduce \underline{O}utlier \underline{R}obust \underline{A}dversarial \underline{T}raining (ORAT) in this work. ORAT is based on a bi-level optimization formulation of adversarial training with a robust rank-based loss function. Theoretically, we show that the learning objective of ORAT satisfies the $\mathcal{H}$-consistency in binary classification, which establishes it as a proper surrogate to adversarial 0/1 loss. Furthermore, we analyze its generalization ability and provide uniform convergence rates in high probability. ORAT can be optimized with a simple algorithm. Experimental evaluations on three benchmark datasets demonstrate the effectiveness and robustness of ORAT in handling outliers and adversarial attacks. Our code is available at \url{https://github.com/discovershu/ORAT}.} }
Endnote
%0 Conference Paper %T Outlier Robust Adversarial Training %A Shu Hu %A Zhenhuan Yang %A Xin Wang %A Yiming Ying %A Siwei Lyu %B Proceedings of the 15th Asian Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Berrin Yanıkoğlu %E Wray Buntine %F pmlr-v222-hu24a %I PMLR %P 454--469 %U https://proceedings.mlr.press/v222/hu24a.html %V 222 %X Supervised learning models are challenged by the intrinsic complexities of training data such as outliers and minority subpopulations and intentional attacks at inference time with adversarial samples. While traditional robust learning methods and the recent adversarial training approaches are designed to handle each of the two challenges, to date, no work has been done to develop models that are robust with regard to the low-quality training data and the potential adversarial attack at inference time simultaneously. It is for this reason that we introduce \underline{O}utlier \underline{R}obust \underline{A}dversarial \underline{T}raining (ORAT) in this work. ORAT is based on a bi-level optimization formulation of adversarial training with a robust rank-based loss function. Theoretically, we show that the learning objective of ORAT satisfies the $\mathcal{H}$-consistency in binary classification, which establishes it as a proper surrogate to adversarial 0/1 loss. Furthermore, we analyze its generalization ability and provide uniform convergence rates in high probability. ORAT can be optimized with a simple algorithm. Experimental evaluations on three benchmark datasets demonstrate the effectiveness and robustness of ORAT in handling outliers and adversarial attacks. Our code is available at \url{https://github.com/discovershu/ORAT}.
APA
Hu, S., Yang, Z., Wang, X., Ying, Y. & Lyu, S.. (2024). Outlier Robust Adversarial Training. Proceedings of the 15th Asian Conference on Machine Learning, in Proceedings of Machine Learning Research 222:454-469 Available from https://proceedings.mlr.press/v222/hu24a.html.

Related Material