Generalization and Equilibrium in Generative Adversarial Nets (GANs)


Sanjeev Arora, Rong Ge, Yingyu Liang, Tengyu Ma, Yi Zhang ;
Proceedings of the 34th International Conference on Machine Learning, PMLR 70:224-232, 2017.


It is shown that training of generative adversarial network (GAN) may not have good generalization properties; e.g., training may appear successful but the trained distribution may be far from target distribution in standard metrics. However, generalization does occur for a weaker metric called neural net distance. It is also shown that an approximate pure equilibrium exists in the discriminator/generator game for a natural training objective (Wasserstein) when generator capacity and training set sizes are moderate. This existence of equilibrium inspires MIX+GAN protocol, which can be combined with any existing GAN training, and empirically shown to improve some of them.

Related Material