A Classification-Based Study of Covariate Shift in GAN Distributions

Shibani Santurkar, Ludwig Schmidt, Aleksander Madry
Proceedings of the 35th International Conference on Machine Learning, PMLR 80:4480-4489, 2018.

Abstract

A basic, and still largely unanswered, question in the context of Generative Adversarial Networks (GANs) is whether they are truly able to capture all the fundamental characteristics of the distributions they are trained on. In particular, evaluating the diversity of GAN distributions is challenging and existing methods provide only a partial understanding of this issue. In this paper, we develop quantitative and scalable tools for assessing the diversity of GAN distributions. Specifically, we take a classification-based perspective and view loss of diversity as a form of covariate shift introduced by GANs. We examine two specific forms of such shift: mode collapse and boundary distortion. In contrast to prior work, our methods need only minimal human supervision and can be readily applied to state-of-the-art GANs on large, canonical datasets. Examining popular GANs using our tools indicates that these GANs have significant problems in reproducing the more distributional properties of their training dataset.

Cite this Paper

BibTeX
@InProceedings{pmlr-v80-santurkar18a, title = {A Classification-Based Study of Covariate Shift in {GAN} Distributions}, author = {Santurkar, Shibani and Schmidt, Ludwig and Madry, Aleksander}, booktitle = {Proceedings of the 35th International Conference on Machine Learning}, pages = {4480--4489}, year = {2018}, editor = {Dy, Jennifer and Krause, Andreas}, volume = {80}, series = {Proceedings of Machine Learning Research}, month = {10--15 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v80/santurkar18a/santurkar18a.pdf}, url = {https://proceedings.mlr.press/v80/santurkar18a.html}, abstract = {A basic, and still largely unanswered, question in the context of Generative Adversarial Networks (GANs) is whether they are truly able to capture all the fundamental characteristics of the distributions they are trained on. In particular, evaluating the diversity of GAN distributions is challenging and existing methods provide only a partial understanding of this issue. In this paper, we develop quantitative and scalable tools for assessing the diversity of GAN distributions. Specifically, we take a classification-based perspective and view loss of diversity as a form of covariate shift introduced by GANs. We examine two specific forms of such shift: mode collapse and boundary distortion. In contrast to prior work, our methods need only minimal human supervision and can be readily applied to state-of-the-art GANs on large, canonical datasets. Examining popular GANs using our tools indicates that these GANs have significant problems in reproducing the more distributional properties of their training dataset.} }
Endnote
%0 Conference Paper %T A Classification-Based Study of Covariate Shift in GAN Distributions %A Shibani Santurkar %A Ludwig Schmidt %A Aleksander Madry %B Proceedings of the 35th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2018 %E Jennifer Dy %E Andreas Krause %F pmlr-v80-santurkar18a %I PMLR %P 4480--4489 %U https://proceedings.mlr.press/v80/santurkar18a.html %V 80 %X A basic, and still largely unanswered, question in the context of Generative Adversarial Networks (GANs) is whether they are truly able to capture all the fundamental characteristics of the distributions they are trained on. In particular, evaluating the diversity of GAN distributions is challenging and existing methods provide only a partial understanding of this issue. In this paper, we develop quantitative and scalable tools for assessing the diversity of GAN distributions. Specifically, we take a classification-based perspective and view loss of diversity as a form of covariate shift introduced by GANs. We examine two specific forms of such shift: mode collapse and boundary distortion. In contrast to prior work, our methods need only minimal human supervision and can be readily applied to state-of-the-art GANs on large, canonical datasets. Examining popular GANs using our tools indicates that these GANs have significant problems in reproducing the more distributional properties of their training dataset.
APA
Santurkar, S., Schmidt, L. & Madry, A.. (2018). A Classification-Based Study of Covariate Shift in GAN Distributions. Proceedings of the 35th International Conference on Machine Learning, in Proceedings of Machine Learning Research 80:4480-4489 Available from https://proceedings.mlr.press/v80/santurkar18a.html.

Related Material