Kernel Mean Matching for Content Addressability of GANs

Wittawat Jitkrittum, Patsorn Sangkloy, Muhammad Waleed Gondal, Amit Raj, James Hays, Bernhard Schölkopf
Proceedings of the 36th International Conference on Machine Learning, PMLR 97:3140-3151, 2019.

Abstract

We propose a novel procedure which adds "content-addressability" to any given unconditional implicit model e.g., a generative adversarial network (GAN). The procedure allows users to control the generative process by specifying a set (arbitrary size) of desired examples based on which similar samples are generated from the model. The proposed approach, based on kernel mean matching, is applicable to any generative models which transform latent vectors to samples, and does not require retraining of the model. Experiments on various high-dimensional image generation problems (CelebA-HQ, LSUN bedroom, bridge, tower) show that our approach is able to generate images which are consistent with the input set, while retaining the image quality of the original model. To our knowledge, this is the first work that attempts to construct, at test time, a content-addressable generative model from a trained marginal model.

Cite this Paper

BibTeX
@InProceedings{pmlr-v97-jitkrittum19a, title = {Kernel Mean Matching for Content Addressability of {GAN}s}, author = {Jitkrittum, Wittawat and Sangkloy, Patsorn and Gondal, Muhammad Waleed and Raj, Amit and Hays, James and Sch{\"o}lkopf, Bernhard}, booktitle = {Proceedings of the 36th International Conference on Machine Learning}, pages = {3140--3151}, year = {2019}, editor = {Chaudhuri, Kamalika and Salakhutdinov, Ruslan}, volume = {97}, series = {Proceedings of Machine Learning Research}, month = {09--15 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v97/jitkrittum19a/jitkrittum19a.pdf}, url = {https://proceedings.mlr.press/v97/jitkrittum19a.html}, abstract = {We propose a novel procedure which adds "content-addressability" to any given unconditional implicit model e.g., a generative adversarial network (GAN). The procedure allows users to control the generative process by specifying a set (arbitrary size) of desired examples based on which similar samples are generated from the model. The proposed approach, based on kernel mean matching, is applicable to any generative models which transform latent vectors to samples, and does not require retraining of the model. Experiments on various high-dimensional image generation problems (CelebA-HQ, LSUN bedroom, bridge, tower) show that our approach is able to generate images which are consistent with the input set, while retaining the image quality of the original model. To our knowledge, this is the first work that attempts to construct, at test time, a content-addressable generative model from a trained marginal model.} }
Endnote
%0 Conference Paper %T Kernel Mean Matching for Content Addressability of GANs %A Wittawat Jitkrittum %A Patsorn Sangkloy %A Muhammad Waleed Gondal %A Amit Raj %A James Hays %A Bernhard Schölkopf %B Proceedings of the 36th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2019 %E Kamalika Chaudhuri %E Ruslan Salakhutdinov %F pmlr-v97-jitkrittum19a %I PMLR %P 3140--3151 %U https://proceedings.mlr.press/v97/jitkrittum19a.html %V 97 %X We propose a novel procedure which adds "content-addressability" to any given unconditional implicit model e.g., a generative adversarial network (GAN). The procedure allows users to control the generative process by specifying a set (arbitrary size) of desired examples based on which similar samples are generated from the model. The proposed approach, based on kernel mean matching, is applicable to any generative models which transform latent vectors to samples, and does not require retraining of the model. Experiments on various high-dimensional image generation problems (CelebA-HQ, LSUN bedroom, bridge, tower) show that our approach is able to generate images which are consistent with the input set, while retaining the image quality of the original model. To our knowledge, this is the first work that attempts to construct, at test time, a content-addressable generative model from a trained marginal model.
APA
Jitkrittum, W., Sangkloy, P., Gondal, M.W., Raj, A., Hays, J. & Schölkopf, B.. (2019). Kernel Mean Matching for Content Addressability of GANs. Proceedings of the 36th International Conference on Machine Learning, in Proceedings of Machine Learning Research 97:3140-3151 Available from https://proceedings.mlr.press/v97/jitkrittum19a.html.

Related Material