Generative Cooperative Networks for Natural Language Generation

Sylvain Lamprier, Thomas Scialom, Antoine Chaffin, Vincent Claveau, Ewa Kijak, Jacopo Staiano, Benjamin Piwowarski
Proceedings of the 39th International Conference on Machine Learning, PMLR 162:11891-11905, 2022.

Abstract

Generative Adversarial Networks (GANs) have known a tremendous success for many continuous generation tasks, especially in the field of image generation. However, for discrete outputs such as language, optimizing GANs remains an open problem with many instabilities, as no gradient can be properly back-propagated from the discriminator output to the generator parameters. An alternative is to learn the generator network via reinforcement learning, using the discriminator signal as a reward, but such a technique suffers from moving rewards and vanishing gradient problems. Finally, it often falls short compared to direct maximum-likelihood approaches. In this paper, we introduce Generative Cooperative Networks, in which the discriminator architecture is cooperatively used along with the generation policy to output samples of realistic texts for the task at hand. We give theoretical guarantees of convergence for our approach, and study various efficient decoding schemes to empirically achieve state-of-the-art results in two main NLG tasks.

Cite this Paper

BibTeX
@InProceedings{pmlr-v162-lamprier22a, title = {Generative Cooperative Networks for Natural Language Generation}, author = {Lamprier, Sylvain and Scialom, Thomas and Chaffin, Antoine and Claveau, Vincent and Kijak, Ewa and Staiano, Jacopo and Piwowarski, Benjamin}, booktitle = {Proceedings of the 39th International Conference on Machine Learning}, pages = {11891--11905}, year = {2022}, editor = {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan}, volume = {162}, series = {Proceedings of Machine Learning Research}, month = {17--23 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v162/lamprier22a/lamprier22a.pdf}, url = {https://proceedings.mlr.press/v162/lamprier22a.html}, abstract = {Generative Adversarial Networks (GANs) have known a tremendous success for many continuous generation tasks, especially in the field of image generation. However, for discrete outputs such as language, optimizing GANs remains an open problem with many instabilities, as no gradient can be properly back-propagated from the discriminator output to the generator parameters. An alternative is to learn the generator network via reinforcement learning, using the discriminator signal as a reward, but such a technique suffers from moving rewards and vanishing gradient problems. Finally, it often falls short compared to direct maximum-likelihood approaches. In this paper, we introduce Generative Cooperative Networks, in which the discriminator architecture is cooperatively used along with the generation policy to output samples of realistic texts for the task at hand. We give theoretical guarantees of convergence for our approach, and study various efficient decoding schemes to empirically achieve state-of-the-art results in two main NLG tasks.} }
Endnote
%0 Conference Paper %T Generative Cooperative Networks for Natural Language Generation %A Sylvain Lamprier %A Thomas Scialom %A Antoine Chaffin %A Vincent Claveau %A Ewa Kijak %A Jacopo Staiano %A Benjamin Piwowarski %B Proceedings of the 39th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2022 %E Kamalika Chaudhuri %E Stefanie Jegelka %E Le Song %E Csaba Szepesvari %E Gang Niu %E Sivan Sabato %F pmlr-v162-lamprier22a %I PMLR %P 11891--11905 %U https://proceedings.mlr.press/v162/lamprier22a.html %V 162 %X Generative Adversarial Networks (GANs) have known a tremendous success for many continuous generation tasks, especially in the field of image generation. However, for discrete outputs such as language, optimizing GANs remains an open problem with many instabilities, as no gradient can be properly back-propagated from the discriminator output to the generator parameters. An alternative is to learn the generator network via reinforcement learning, using the discriminator signal as a reward, but such a technique suffers from moving rewards and vanishing gradient problems. Finally, it often falls short compared to direct maximum-likelihood approaches. In this paper, we introduce Generative Cooperative Networks, in which the discriminator architecture is cooperatively used along with the generation policy to output samples of realistic texts for the task at hand. We give theoretical guarantees of convergence for our approach, and study various efficient decoding schemes to empirically achieve state-of-the-art results in two main NLG tasks.
APA
Lamprier, S., Scialom, T., Chaffin, A., Claveau, V., Kijak, E., Staiano, J. & Piwowarski, B.. (2022). Generative Cooperative Networks for Natural Language Generation. Proceedings of the 39th International Conference on Machine Learning, in Proceedings of Machine Learning Research 162:11891-11905 Available from https://proceedings.mlr.press/v162/lamprier22a.html.

Related Material