Negative Momentum for Improved Game Dynamics

Gauthier Gidel, Reyhane Askari Hemmat, Mohammad Pezeshki, Rémi Le Priol, Gabriel Huang, Simon Lacoste-Julien, Ioannis Mitliagkas
; Proceedings of Machine Learning Research, PMLR 89:1802-1811, 2019.

Abstract

Games generalize the single-objective optimization paradigm by introducing different objective functions for different players. Differentiable games often proceed by simultaneous or alternating gradient updates. In machine learning, games are gaining new importance through formulations like generative adversarial networks (GANs) and actor-critic systems. However, compared to single-objective optimization, game dynamics is more complex and less understood. In this paper, we analyze gradient-based methods with momentum on simple games. We prove that alternating updates are more stable than simultaneous updates. Next, we show both theoretically and empirically that alternating gradient updates with a negative momentum term achieves convergence in a difficult toy adversarial problem, but also on the notoriously difficult to train saturating GANs.

Cite this Paper


BibTeX
@InProceedings{pmlr-v89-gidel19a, title = {Negative Momentum for Improved Game Dynamics}, author = {Gidel, Gauthier and Hemmat, Reyhane Askari and Pezeshki, Mohammad and Priol, R\'emi Le and Huang, Gabriel and Lacoste-Julien, Simon and Mitliagkas, Ioannis}, booktitle = {Proceedings of Machine Learning Research}, pages = {1802--1811}, year = {2019}, editor = {Kamalika Chaudhuri and Masashi Sugiyama}, volume = {89}, series = {Proceedings of Machine Learning Research}, address = {}, month = {16--18 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v89/gidel19a/gidel19a.pdf}, url = {http://proceedings.mlr.press/v89/gidel19a.html}, abstract = {Games generalize the single-objective optimization paradigm by introducing different objective functions for different players. Differentiable games often proceed by simultaneous or alternating gradient updates. In machine learning, games are gaining new importance through formulations like generative adversarial networks (GANs) and actor-critic systems. However, compared to single-objective optimization, game dynamics is more complex and less understood. In this paper, we analyze gradient-based methods with momentum on simple games. We prove that alternating updates are more stable than simultaneous updates. Next, we show both theoretically and empirically that alternating gradient updates with a negative momentum term achieves convergence in a difficult toy adversarial problem, but also on the notoriously difficult to train saturating GANs.} }
Endnote
%0 Conference Paper %T Negative Momentum for Improved Game Dynamics %A Gauthier Gidel %A Reyhane Askari Hemmat %A Mohammad Pezeshki %A Rémi Le Priol %A Gabriel Huang %A Simon Lacoste-Julien %A Ioannis Mitliagkas %B Proceedings of Machine Learning Research %C Proceedings of Machine Learning Research %D 2019 %E Kamalika Chaudhuri %E Masashi Sugiyama %F pmlr-v89-gidel19a %I PMLR %J Proceedings of Machine Learning Research %P 1802--1811 %U http://proceedings.mlr.press %V 89 %W PMLR %X Games generalize the single-objective optimization paradigm by introducing different objective functions for different players. Differentiable games often proceed by simultaneous or alternating gradient updates. In machine learning, games are gaining new importance through formulations like generative adversarial networks (GANs) and actor-critic systems. However, compared to single-objective optimization, game dynamics is more complex and less understood. In this paper, we analyze gradient-based methods with momentum on simple games. We prove that alternating updates are more stable than simultaneous updates. Next, we show both theoretically and empirically that alternating gradient updates with a negative momentum term achieves convergence in a difficult toy adversarial problem, but also on the notoriously difficult to train saturating GANs.
APA
Gidel, G., Hemmat, R.A., Pezeshki, M., Priol, R.L., Huang, G., Lacoste-Julien, S. & Mitliagkas, I.. (2019). Negative Momentum for Improved Game Dynamics. Proceedings of Machine Learning Research, in PMLR 89:1802-1811

Related Material