Generative Adversarial Text to Image Synthesis

Scott Reed, Zeynep Akata, Xinchen Yan, Lajanugen Logeswaran, Bernt Schiele, Honglak Lee
Proceedings of The 33rd International Conference on Machine Learning, PMLR 48:1060-1069, 2016.

Abstract

Automatic synthesis of realistic images from text would be interesting and useful, but current AI systems are still far from this goal. However, in recent years generic and powerful recurrent neural network architectures have been developed to learn discriminative text feature representations. Meanwhile, deep convolutional generative adversarial networks (GANs) have begun to generate highly compelling images of specific categories such as faces, album covers, room interiors and flowers. In this work, we develop a novel deep architecture and GAN formulation to effectively bridge these advances in text and image modeling, translating visual concepts from characters to pixels. We demonstrate the capability of our model to generate plausible images of birds and flowers from detailed text descriptions.

Cite this Paper

BibTeX
@InProceedings{pmlr-v48-reed16, title = {Generative Adversarial Text to Image Synthesis}, author = {Reed, Scott and Akata, Zeynep and Yan, Xinchen and Logeswaran, Lajanugen and Schiele, Bernt and Lee, Honglak}, booktitle = {Proceedings of The 33rd International Conference on Machine Learning}, pages = {1060--1069}, year = {2016}, editor = {Balcan, Maria Florina and Weinberger, Kilian Q.}, volume = {48}, series = {Proceedings of Machine Learning Research}, address = {New York, New York, USA}, month = {20--22 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v48/reed16.pdf}, url = {https://proceedings.mlr.press/v48/reed16.html}, abstract = {Automatic synthesis of realistic images from text would be interesting and useful, but current AI systems are still far from this goal. However, in recent years generic and powerful recurrent neural network architectures have been developed to learn discriminative text feature representations. Meanwhile, deep convolutional generative adversarial networks (GANs) have begun to generate highly compelling images of specific categories such as faces, album covers, room interiors and flowers. In this work, we develop a novel deep architecture and GAN formulation to effectively bridge these advances in text and image modeling, translating visual concepts from characters to pixels. We demonstrate the capability of our model to generate plausible images of birds and flowers from detailed text descriptions.} }
Endnote
%0 Conference Paper %T Generative Adversarial Text to Image Synthesis %A Scott Reed %A Zeynep Akata %A Xinchen Yan %A Lajanugen Logeswaran %A Bernt Schiele %A Honglak Lee %B Proceedings of The 33rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2016 %E Maria Florina Balcan %E Kilian Q. Weinberger %F pmlr-v48-reed16 %I PMLR %P 1060--1069 %U https://proceedings.mlr.press/v48/reed16.html %V 48 %X Automatic synthesis of realistic images from text would be interesting and useful, but current AI systems are still far from this goal. However, in recent years generic and powerful recurrent neural network architectures have been developed to learn discriminative text feature representations. Meanwhile, deep convolutional generative adversarial networks (GANs) have begun to generate highly compelling images of specific categories such as faces, album covers, room interiors and flowers. In this work, we develop a novel deep architecture and GAN formulation to effectively bridge these advances in text and image modeling, translating visual concepts from characters to pixels. We demonstrate the capability of our model to generate plausible images of birds and flowers from detailed text descriptions.
RIS
TY - CPAPER TI - Generative Adversarial Text to Image Synthesis AU - Scott Reed AU - Zeynep Akata AU - Xinchen Yan AU - Lajanugen Logeswaran AU - Bernt Schiele AU - Honglak Lee BT - Proceedings of The 33rd International Conference on Machine Learning DA - 2016/06/11 ED - Maria Florina Balcan ED - Kilian Q. Weinberger ID - pmlr-v48-reed16 PB - PMLR DP - Proceedings of Machine Learning Research VL - 48 SP - 1060 EP - 1069 L1 - http://proceedings.mlr.press/v48/reed16.pdf UR - https://proceedings.mlr.press/v48/reed16.html AB - Automatic synthesis of realistic images from text would be interesting and useful, but current AI systems are still far from this goal. However, in recent years generic and powerful recurrent neural network architectures have been developed to learn discriminative text feature representations. Meanwhile, deep convolutional generative adversarial networks (GANs) have begun to generate highly compelling images of specific categories such as faces, album covers, room interiors and flowers. In this work, we develop a novel deep architecture and GAN formulation to effectively bridge these advances in text and image modeling, translating visual concepts from characters to pixels. We demonstrate the capability of our model to generate plausible images of birds and flowers from detailed text descriptions. ER -
APA
Reed, S., Akata, Z., Yan, X., Logeswaran, L., Schiele, B. & Lee, H.. (2016). Generative Adversarial Text to Image Synthesis. Proceedings of The 33rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 48:1060-1069 Available from https://proceedings.mlr.press/v48/reed16.html.

Related Material