A Large-Scale Study on Regularization and Normalization in GANs

Karol Kurach, Mario Lučić, Xiaohua Zhai, Marcin Michalski, Sylvain Gelly
Proceedings of the 36th International Conference on Machine Learning, PMLR 97:3581-3590, 2019.

Abstract

Generative adversarial networks (GANs) are a class of deep generative models which aim to learn a target distribution in an unsupervised fashion. While they were successfully applied to many problems, training a GAN is a notoriously challenging task and requires a significant number of hyperparameter tuning, neural architecture engineering, and a non-trivial amount of “tricks". The success in many practical applications coupled with the lack of a measure to quantify the failure modes of GANs resulted in a plethora of proposed losses, regularization and normalization schemes, as well as neural architectures. In this work we take a sober view of the current state of GANs from a practical perspective. We discuss and evaluate common pitfalls and reproducibility issues, open-source our code on Github, and provide pre-trained models on TensorFlow Hub.

Cite this Paper

BibTeX
@InProceedings{pmlr-v97-kurach19a, title = {A Large-Scale Study on Regularization and Normalization in {GAN}s}, author = {Kurach, Karol and Lu{\v{c}}i{\'c}, Mario and Zhai, Xiaohua and Michalski, Marcin and Gelly, Sylvain}, booktitle = {Proceedings of the 36th International Conference on Machine Learning}, pages = {3581--3590}, year = {2019}, editor = {Chaudhuri, Kamalika and Salakhutdinov, Ruslan}, volume = {97}, series = {Proceedings of Machine Learning Research}, month = {09--15 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v97/kurach19a/kurach19a.pdf}, url = {https://proceedings.mlr.press/v97/kurach19a.html}, abstract = {Generative adversarial networks (GANs) are a class of deep generative models which aim to learn a target distribution in an unsupervised fashion. While they were successfully applied to many problems, training a GAN is a notoriously challenging task and requires a significant number of hyperparameter tuning, neural architecture engineering, and a non-trivial amount of “tricks". The success in many practical applications coupled with the lack of a measure to quantify the failure modes of GANs resulted in a plethora of proposed losses, regularization and normalization schemes, as well as neural architectures. In this work we take a sober view of the current state of GANs from a practical perspective. We discuss and evaluate common pitfalls and reproducibility issues, open-source our code on Github, and provide pre-trained models on TensorFlow Hub.} }
Endnote
%0 Conference Paper %T A Large-Scale Study on Regularization and Normalization in GANs %A Karol Kurach %A Mario Lučić %A Xiaohua Zhai %A Marcin Michalski %A Sylvain Gelly %B Proceedings of the 36th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2019 %E Kamalika Chaudhuri %E Ruslan Salakhutdinov %F pmlr-v97-kurach19a %I PMLR %P 3581--3590 %U https://proceedings.mlr.press/v97/kurach19a.html %V 97 %X Generative adversarial networks (GANs) are a class of deep generative models which aim to learn a target distribution in an unsupervised fashion. While they were successfully applied to many problems, training a GAN is a notoriously challenging task and requires a significant number of hyperparameter tuning, neural architecture engineering, and a non-trivial amount of “tricks". The success in many practical applications coupled with the lack of a measure to quantify the failure modes of GANs resulted in a plethora of proposed losses, regularization and normalization schemes, as well as neural architectures. In this work we take a sober view of the current state of GANs from a practical perspective. We discuss and evaluate common pitfalls and reproducibility issues, open-source our code on Github, and provide pre-trained models on TensorFlow Hub.
APA
Kurach, K., Lučić, M., Zhai, X., Michalski, M. & Gelly, S.. (2019). A Large-Scale Study on Regularization and Normalization in GANs. Proceedings of the 36th International Conference on Machine Learning, in Proceedings of Machine Learning Research 97:3581-3590 Available from https://proceedings.mlr.press/v97/kurach19a.html.

Related Material