Unsupervised Discovery of Interpretable Directions in the GAN Latent Space

Andrey Voynov, Artem Babenko
Proceedings of the 37th International Conference on Machine Learning, PMLR 119:9786-9796, 2020.

Abstract

The latent spaces of GAN models often have semantically meaningful directions. Moving in these directions corresponds to human-interpretable image transformations, such as zooming or recoloring, enabling a more controllable generation process. However, the discovery of such directions is currently performed in a supervised manner, requiring human labels, pretrained models, or some form of self-supervision. These requirements severely restrict a range of directions existing approaches can discover. In this paper, we introduce an unsupervised method to identify interpretable directions in the latent space of a pretrained GAN model. By a simple model-agnostic procedure, we find directions corresponding to sensible semantic manipulations without any form of (self-)supervision. Furthermore, we reveal several non-trivial findings, which would be difficult to obtain by existing methods, e.g., a direction corresponding to background removal. As an immediate practical benefit of our work, we show how to exploit this finding to achieve competitive performance for weakly-supervised saliency detection. The implementation of our method is available online.

Cite this Paper

BibTeX
@InProceedings{pmlr-v119-voynov20a, title = {Unsupervised Discovery of Interpretable Directions in the {GAN} Latent Space}, author = {Voynov, Andrey and Babenko, Artem}, booktitle = {Proceedings of the 37th International Conference on Machine Learning}, pages = {9786--9796}, year = {2020}, editor = {III, Hal Daumé and Singh, Aarti}, volume = {119}, series = {Proceedings of Machine Learning Research}, month = {13--18 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v119/voynov20a/voynov20a.pdf}, url = {https://proceedings.mlr.press/v119/voynov20a.html}, abstract = {The latent spaces of GAN models often have semantically meaningful directions. Moving in these directions corresponds to human-interpretable image transformations, such as zooming or recoloring, enabling a more controllable generation process. However, the discovery of such directions is currently performed in a supervised manner, requiring human labels, pretrained models, or some form of self-supervision. These requirements severely restrict a range of directions existing approaches can discover. In this paper, we introduce an unsupervised method to identify interpretable directions in the latent space of a pretrained GAN model. By a simple model-agnostic procedure, we find directions corresponding to sensible semantic manipulations without any form of (self-)supervision. Furthermore, we reveal several non-trivial findings, which would be difficult to obtain by existing methods, e.g., a direction corresponding to background removal. As an immediate practical benefit of our work, we show how to exploit this finding to achieve competitive performance for weakly-supervised saliency detection. The implementation of our method is available online.} }
Endnote
%0 Conference Paper %T Unsupervised Discovery of Interpretable Directions in the GAN Latent Space %A Andrey Voynov %A Artem Babenko %B Proceedings of the 37th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2020 %E Hal Daumé III %E Aarti Singh %F pmlr-v119-voynov20a %I PMLR %P 9786--9796 %U https://proceedings.mlr.press/v119/voynov20a.html %V 119 %X The latent spaces of GAN models often have semantically meaningful directions. Moving in these directions corresponds to human-interpretable image transformations, such as zooming or recoloring, enabling a more controllable generation process. However, the discovery of such directions is currently performed in a supervised manner, requiring human labels, pretrained models, or some form of self-supervision. These requirements severely restrict a range of directions existing approaches can discover. In this paper, we introduce an unsupervised method to identify interpretable directions in the latent space of a pretrained GAN model. By a simple model-agnostic procedure, we find directions corresponding to sensible semantic manipulations without any form of (self-)supervision. Furthermore, we reveal several non-trivial findings, which would be difficult to obtain by existing methods, e.g., a direction corresponding to background removal. As an immediate practical benefit of our work, we show how to exploit this finding to achieve competitive performance for weakly-supervised saliency detection. The implementation of our method is available online.
APA
Voynov, A. & Babenko, A.. (2020). Unsupervised Discovery of Interpretable Directions in the GAN Latent Space. Proceedings of the 37th International Conference on Machine Learning, in Proceedings of Machine Learning Research 119:9786-9796 Available from https://proceedings.mlr.press/v119/voynov20a.html.

Related Material