On Minimax Optimality of GANs for Robust Mean Estimation

Kaiwen Wu, Gavin Weiguang Ding, Ruitong Huang, Yaoliang Yu
Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, PMLR 108:4541-4551, 2020.

Abstract

Generative adversarial networks (GANs) have become one of the most popular generative modeling techniques in machine learning. In this work, we study the statistical and robust properties of GANs for Gaussian mean estimation under Huber’s contamination model, where an epsilon proportion of training data may be arbitrarily corrupted. We prove that f-GAN, when equipped with appropriate discriminators, achieve optimal minimax rate, hence extending the recent result of Gao et al. (2019a). In contrast, we show that other GAN variants such as MMD-GAN (with Gaussian kernel) and W-GAN may fail to achieve minimax optimality. We further adapt f-GAN to the sparse and the unknown covariance settings. We perform numerical simulations to confirm our theoretical findings and reveal new insights on the importance of discriminators.

Cite this Paper

BibTeX
@InProceedings{pmlr-v108-wu20d, title = {On Minimax Optimality of GANs for Robust Mean Estimation}, author = {Wu, Kaiwen and Ding, Gavin Weiguang and Huang, Ruitong and Yu, Yaoliang}, booktitle = {Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics}, pages = {4541--4551}, year = {2020}, editor = {Chiappa, Silvia and Calandra, Roberto}, volume = {108}, series = {Proceedings of Machine Learning Research}, month = {26--28 Aug}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v108/wu20d/wu20d.pdf}, url = {https://proceedings.mlr.press/v108/wu20d.html}, abstract = {Generative adversarial networks (GANs) have become one of the most popular generative modeling techniques in machine learning. In this work, we study the statistical and robust properties of GANs for Gaussian mean estimation under Huber’s contamination model, where an epsilon proportion of training data may be arbitrarily corrupted. We prove that f-GAN, when equipped with appropriate discriminators, achieve optimal minimax rate, hence extending the recent result of Gao et al. (2019a). In contrast, we show that other GAN variants such as MMD-GAN (with Gaussian kernel) and W-GAN may fail to achieve minimax optimality. We further adapt f-GAN to the sparse and the unknown covariance settings. We perform numerical simulations to confirm our theoretical findings and reveal new insights on the importance of discriminators.} }
Endnote
%0 Conference Paper %T On Minimax Optimality of GANs for Robust Mean Estimation %A Kaiwen Wu %A Gavin Weiguang Ding %A Ruitong Huang %A Yaoliang Yu %B Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2020 %E Silvia Chiappa %E Roberto Calandra %F pmlr-v108-wu20d %I PMLR %P 4541--4551 %U https://proceedings.mlr.press/v108/wu20d.html %V 108 %X Generative adversarial networks (GANs) have become one of the most popular generative modeling techniques in machine learning. In this work, we study the statistical and robust properties of GANs for Gaussian mean estimation under Huber’s contamination model, where an epsilon proportion of training data may be arbitrarily corrupted. We prove that f-GAN, when equipped with appropriate discriminators, achieve optimal minimax rate, hence extending the recent result of Gao et al. (2019a). In contrast, we show that other GAN variants such as MMD-GAN (with Gaussian kernel) and W-GAN may fail to achieve minimax optimality. We further adapt f-GAN to the sparse and the unknown covariance settings. We perform numerical simulations to confirm our theoretical findings and reveal new insights on the importance of discriminators.
APA
Wu, K., Ding, G.W., Huang, R. & Yu, Y.. (2020). On Minimax Optimality of GANs for Robust Mean Estimation. Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 108:4541-4551 Available from https://proceedings.mlr.press/v108/wu20d.html.

Related Material