WGAN with an Infinitely Wide Generator Has No Spurious Stationary Points

Albert No, Taeho Yoon, Kwon Sehyun, Ernest K Ryu
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:8205-8215, 2021.

Abstract

Generative adversarial networks (GAN) are a widely used class of deep generative models, but their minimax training dynamics are not understood very well. In this work, we show that GANs with a 2-layer infinite-width generator and a 2-layer finite-width discriminator trained with stochastic gradient ascent-descent have no spurious stationary points. We then show that when the width of the generator is finite but wide, there are no spurious stationary points within a ball whose radius becomes arbitrarily large (to cover the entire parameter space) as the width goes to infinity.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-no21a, title = {WGAN with an Infinitely Wide Generator Has No Spurious Stationary Points}, author = {No, Albert and Yoon, Taeho and Sehyun, Kwon and Ryu, Ernest K}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {8205--8215}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/no21a/no21a.pdf}, url = {https://proceedings.mlr.press/v139/no21a.html}, abstract = {Generative adversarial networks (GAN) are a widely used class of deep generative models, but their minimax training dynamics are not understood very well. In this work, we show that GANs with a 2-layer infinite-width generator and a 2-layer finite-width discriminator trained with stochastic gradient ascent-descent have no spurious stationary points. We then show that when the width of the generator is finite but wide, there are no spurious stationary points within a ball whose radius becomes arbitrarily large (to cover the entire parameter space) as the width goes to infinity.} }
Endnote
%0 Conference Paper %T WGAN with an Infinitely Wide Generator Has No Spurious Stationary Points %A Albert No %A Taeho Yoon %A Kwon Sehyun %A Ernest K Ryu %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-no21a %I PMLR %P 8205--8215 %U https://proceedings.mlr.press/v139/no21a.html %V 139 %X Generative adversarial networks (GAN) are a widely used class of deep generative models, but their minimax training dynamics are not understood very well. In this work, we show that GANs with a 2-layer infinite-width generator and a 2-layer finite-width discriminator trained with stochastic gradient ascent-descent have no spurious stationary points. We then show that when the width of the generator is finite but wide, there are no spurious stationary points within a ball whose radius becomes arbitrarily large (to cover the entire parameter space) as the width goes to infinity.
APA
No, A., Yoon, T., Sehyun, K. & Ryu, E.K.. (2021). WGAN with an Infinitely Wide Generator Has No Spurious Stationary Points. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:8205-8215 Available from https://proceedings.mlr.press/v139/no21a.html.

Related Material