Unlabeled Data Help: Minimax Analysis and Adversarial Robustness

Yue Xing, Qifan Song, Guang Cheng
Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, PMLR 151:136-168, 2022.

Abstract

The recent proposed self-supervised learning (SSL) approaches successfully demonstrate the great potential of supplementing learning algorithms with additional unlabeled data. However, it is still unclear whether the existing SSL algorithms can fully utilize the information of both labelled and unlabeled data. This paper gives an affirmative answer for the reconstruction-based SSL algorithm (Lee et al., 2020) under several statistical models. While existing literature only focuses on establishing the upper bound of the convergence rate, we provide a rigorous minimax analysis, and successfully justify the rate-optimality of the reconstruction-based SSL algorithm under different data generation models. Furthermore, we incorporate the reconstruction-based SSL into the exist- ing adversarial training algorithms and show that learning from unlabeled data helps improve the robustness.

Cite this Paper

BibTeX
@InProceedings{pmlr-v151-xing22a, title = { Unlabeled Data Help: Minimax Analysis and Adversarial Robustness }, author = {Xing, Yue and Song, Qifan and Cheng, Guang}, booktitle = {Proceedings of The 25th International Conference on Artificial Intelligence and Statistics}, pages = {136--168}, year = {2022}, editor = {Camps-Valls, Gustau and Ruiz, Francisco J. R. and Valera, Isabel}, volume = {151}, series = {Proceedings of Machine Learning Research}, month = {28--30 Mar}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v151/xing22a/xing22a.pdf}, url = {https://proceedings.mlr.press/v151/xing22a.html}, abstract = { The recent proposed self-supervised learning (SSL) approaches successfully demonstrate the great potential of supplementing learning algorithms with additional unlabeled data. However, it is still unclear whether the existing SSL algorithms can fully utilize the information of both labelled and unlabeled data. This paper gives an affirmative answer for the reconstruction-based SSL algorithm (Lee et al., 2020) under several statistical models. While existing literature only focuses on establishing the upper bound of the convergence rate, we provide a rigorous minimax analysis, and successfully justify the rate-optimality of the reconstruction-based SSL algorithm under different data generation models. Furthermore, we incorporate the reconstruction-based SSL into the exist- ing adversarial training algorithms and show that learning from unlabeled data helps improve the robustness. } }
Endnote
%0 Conference Paper %T Unlabeled Data Help: Minimax Analysis and Adversarial Robustness %A Yue Xing %A Qifan Song %A Guang Cheng %B Proceedings of The 25th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2022 %E Gustau Camps-Valls %E Francisco J. R. Ruiz %E Isabel Valera %F pmlr-v151-xing22a %I PMLR %P 136--168 %U https://proceedings.mlr.press/v151/xing22a.html %V 151 %X The recent proposed self-supervised learning (SSL) approaches successfully demonstrate the great potential of supplementing learning algorithms with additional unlabeled data. However, it is still unclear whether the existing SSL algorithms can fully utilize the information of both labelled and unlabeled data. This paper gives an affirmative answer for the reconstruction-based SSL algorithm (Lee et al., 2020) under several statistical models. While existing literature only focuses on establishing the upper bound of the convergence rate, we provide a rigorous minimax analysis, and successfully justify the rate-optimality of the reconstruction-based SSL algorithm under different data generation models. Furthermore, we incorporate the reconstruction-based SSL into the exist- ing adversarial training algorithms and show that learning from unlabeled data helps improve the robustness.
APA
Xing, Y., Song, Q. & Cheng, G.. (2022). Unlabeled Data Help: Minimax Analysis and Adversarial Robustness . Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 151:136-168 Available from https://proceedings.mlr.press/v151/xing22a.html.

Related Material