Adversarial Discrete Sequence Generation without Explicit NeuralNetworks as Discriminators

Zhongliang Li, Tian Xia, Xingyu Lou, Kaihe Xu, Shaojun Wang, Jing Xiao
Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics, PMLR 89:3089-3098, 2019.

Abstract

This paper presents a novel approach to train GANs for discrete sequence generation without resorting to an explicit neural network as the discriminator. We show that when an alternative mini-max optimization procedure is performed for the value function where a closed form solution for the discriminator exists in the maximization step, it is equivalent to directly optimizing the Jenson-Shannon divergence (JSD) between the generator’s distribution and the empirical distribution over the training data without sampling from the generator, thus optimizing the JSD becomes computationally tractable to train the generator that generates sequences of discrete data. Extensive experiments on synthetic data and real-world tasks demonstrate significant improvements over existing methods to train GANs that generate discrete sequences.

Cite this Paper

BibTeX
@InProceedings{pmlr-v89-li19g, title = {Adversarial Discrete Sequence Generation without Explicit NeuralNetworks as Discriminators}, author = {Li, Zhongliang and Xia, Tian and Lou, Xingyu and Xu, Kaihe and Wang, Shaojun and Xiao, Jing}, booktitle = {Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics}, pages = {3089--3098}, year = {2019}, editor = {Chaudhuri, Kamalika and Sugiyama, Masashi}, volume = {89}, series = {Proceedings of Machine Learning Research}, month = {16--18 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v89/li19g/li19g.pdf}, url = {https://proceedings.mlr.press/v89/li19g.html}, abstract = {This paper presents a novel approach to train GANs for discrete sequence generation without resorting to an explicit neural network as the discriminator. We show that when an alternative mini-max optimization procedure is performed for the value function where a closed form solution for the discriminator exists in the maximization step, it is equivalent to directly optimizing the Jenson-Shannon divergence (JSD) between the generator’s distribution and the empirical distribution over the training data without sampling from the generator, thus optimizing the JSD becomes computationally tractable to train the generator that generates sequences of discrete data. Extensive experiments on synthetic data and real-world tasks demonstrate significant improvements over existing methods to train GANs that generate discrete sequences.} }
Endnote
%0 Conference Paper %T Adversarial Discrete Sequence Generation without Explicit NeuralNetworks as Discriminators %A Zhongliang Li %A Tian Xia %A Xingyu Lou %A Kaihe Xu %A Shaojun Wang %A Jing Xiao %B Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2019 %E Kamalika Chaudhuri %E Masashi Sugiyama %F pmlr-v89-li19g %I PMLR %P 3089--3098 %U https://proceedings.mlr.press/v89/li19g.html %V 89 %X This paper presents a novel approach to train GANs for discrete sequence generation without resorting to an explicit neural network as the discriminator. We show that when an alternative mini-max optimization procedure is performed for the value function where a closed form solution for the discriminator exists in the maximization step, it is equivalent to directly optimizing the Jenson-Shannon divergence (JSD) between the generator’s distribution and the empirical distribution over the training data without sampling from the generator, thus optimizing the JSD becomes computationally tractable to train the generator that generates sequences of discrete data. Extensive experiments on synthetic data and real-world tasks demonstrate significant improvements over existing methods to train GANs that generate discrete sequences.
APA
Li, Z., Xia, T., Lou, X., Xu, K., Wang, S. & Xiao, J.. (2019). Adversarial Discrete Sequence Generation without Explicit NeuralNetworks as Discriminators. Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 89:3089-3098 Available from https://proceedings.mlr.press/v89/li19g.html.

Related Material