Semi-Supervised StyleGAN for Disentanglement Learning

Weili Nie, Tero Karras, Animesh Garg, Shoubhik Debnath, Anjul Patney, Ankit Patel, Animashree Anandkumar
Proceedings of the 37th International Conference on Machine Learning, PMLR 119:7360-7369, 2020.

Abstract

Disentanglement learning is crucial for obtaining disentangled representations and controllable generation. Current disentanglement methods face several inherent limitations: difficulty with high-resolution images, primarily focusing on learning disentangled representations, and non-identifiability due to the unsupervised setting. To alleviate these limitations, we design new architectures and loss functions based on StyleGAN (Karras et al., 2019), for semi-supervised high-resolution disentanglement learning. We create two complex high-resolution synthetic datasets for systematic testing. We investigate the impact of limited supervision and find that using only 0.25% 2.5% of labeled data is sufficient for good disentanglement on both synthetic and real datasets. We propose new metrics to quantify generator controllability, and observe there may exist a crucial trade-off between disentangled representation learning and controllable generation. We also consider semantic fine-grained image editing to achieve better generalization to unseen images.

Cite this Paper

BibTeX
@InProceedings{pmlr-v119-nie20a, title = {Semi-Supervised {S}tyle{GAN} for Disentanglement Learning}, author = {Nie, Weili and Karras, Tero and Garg, Animesh and Debnath, Shoubhik and Patney, Anjul and Patel, Ankit and Anandkumar, Animashree}, booktitle = {Proceedings of the 37th International Conference on Machine Learning}, pages = {7360--7369}, year = {2020}, editor = {III, Hal Daumé and Singh, Aarti}, volume = {119}, series = {Proceedings of Machine Learning Research}, month = {13--18 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v119/nie20a/nie20a.pdf}, url = {https://proceedings.mlr.press/v119/nie20a.html}, abstract = {Disentanglement learning is crucial for obtaining disentangled representations and controllable generation. Current disentanglement methods face several inherent limitations: difficulty with high-resolution images, primarily focusing on learning disentangled representations, and non-identifiability due to the unsupervised setting. To alleviate these limitations, we design new architectures and loss functions based on StyleGAN (Karras et al., 2019), for semi-supervised high-resolution disentanglement learning. We create two complex high-resolution synthetic datasets for systematic testing. We investigate the impact of limited supervision and find that using only 0.25% 2.5% of labeled data is sufficient for good disentanglement on both synthetic and real datasets. We propose new metrics to quantify generator controllability, and observe there may exist a crucial trade-off between disentangled representation learning and controllable generation. We also consider semantic fine-grained image editing to achieve better generalization to unseen images.} }
Endnote
%0 Conference Paper %T Semi-Supervised StyleGAN for Disentanglement Learning %A Weili Nie %A Tero Karras %A Animesh Garg %A Shoubhik Debnath %A Anjul Patney %A Ankit Patel %A Animashree Anandkumar %B Proceedings of the 37th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2020 %E Hal Daumé III %E Aarti Singh %F pmlr-v119-nie20a %I PMLR %P 7360--7369 %U https://proceedings.mlr.press/v119/nie20a.html %V 119 %X Disentanglement learning is crucial for obtaining disentangled representations and controllable generation. Current disentanglement methods face several inherent limitations: difficulty with high-resolution images, primarily focusing on learning disentangled representations, and non-identifiability due to the unsupervised setting. To alleviate these limitations, we design new architectures and loss functions based on StyleGAN (Karras et al., 2019), for semi-supervised high-resolution disentanglement learning. We create two complex high-resolution synthetic datasets for systematic testing. We investigate the impact of limited supervision and find that using only 0.25% 2.5% of labeled data is sufficient for good disentanglement on both synthetic and real datasets. We propose new metrics to quantify generator controllability, and observe there may exist a crucial trade-off between disentangled representation learning and controllable generation. We also consider semantic fine-grained image editing to achieve better generalization to unseen images.
APA
Nie, W., Karras, T., Garg, A., Debnath, S., Patney, A., Patel, A. & Anandkumar, A.. (2020). Semi-Supervised StyleGAN for Disentanglement Learning. Proceedings of the 37th International Conference on Machine Learning, in Proceedings of Machine Learning Research 119:7360-7369 Available from https://proceedings.mlr.press/v119/nie20a.html.

Related Material