Neural Optimizer Search with Reinforcement Learning

Irwan Bello, Barret Zoph, Vijay Vasudevan, Quoc V. Le
Proceedings of the 34th International Conference on Machine Learning, PMLR 70:459-468, 2017.

Abstract

We present an approach to automate the process of discovering optimization methods, with a focus on deep learning architectures. We train a Recurrent Neural Network controller to generate a string in a specific domain language that describes a mathematical update equation based on a list of primitive functions, such as the gradient, running average of the gradient, etc. The controller is trained with Reinforcement Learning to maximize the performance of a model after a few epochs. On CIFAR-10, our method discovers several update rules that are better than many commonly used optimizers, such as Adam, RMSProp, or SGD with and without Momentum on a ConvNet model. These optimizers can also be transferred to perform well on different neural network architectures, including Google’s neural machine translation system.

Cite this Paper

BibTeX
@InProceedings{pmlr-v70-bello17a, title = {Neural Optimizer Search with Reinforcement Learning}, author = {Irwan Bello and Barret Zoph and Vijay Vasudevan and Quoc V. Le}, booktitle = {Proceedings of the 34th International Conference on Machine Learning}, pages = {459--468}, year = {2017}, editor = {Precup, Doina and Teh, Yee Whye}, volume = {70}, series = {Proceedings of Machine Learning Research}, month = {06--11 Aug}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v70/bello17a/bello17a.pdf}, url = {https://proceedings.mlr.press/v70/bello17a.html}, abstract = {We present an approach to automate the process of discovering optimization methods, with a focus on deep learning architectures. We train a Recurrent Neural Network controller to generate a string in a specific domain language that describes a mathematical update equation based on a list of primitive functions, such as the gradient, running average of the gradient, etc. The controller is trained with Reinforcement Learning to maximize the performance of a model after a few epochs. On CIFAR-10, our method discovers several update rules that are better than many commonly used optimizers, such as Adam, RMSProp, or SGD with and without Momentum on a ConvNet model. These optimizers can also be transferred to perform well on different neural network architectures, including Google’s neural machine translation system.} }
Endnote
%0 Conference Paper %T Neural Optimizer Search with Reinforcement Learning %A Irwan Bello %A Barret Zoph %A Vijay Vasudevan %A Quoc V. Le %B Proceedings of the 34th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2017 %E Doina Precup %E Yee Whye Teh %F pmlr-v70-bello17a %I PMLR %P 459--468 %U https://proceedings.mlr.press/v70/bello17a.html %V 70 %X We present an approach to automate the process of discovering optimization methods, with a focus on deep learning architectures. We train a Recurrent Neural Network controller to generate a string in a specific domain language that describes a mathematical update equation based on a list of primitive functions, such as the gradient, running average of the gradient, etc. The controller is trained with Reinforcement Learning to maximize the performance of a model after a few epochs. On CIFAR-10, our method discovers several update rules that are better than many commonly used optimizers, such as Adam, RMSProp, or SGD with and without Momentum on a ConvNet model. These optimizers can also be transferred to perform well on different neural network architectures, including Google’s neural machine translation system.
APA
Bello, I., Zoph, B., Vasudevan, V. & Le, Q.V.. (2017). Neural Optimizer Search with Reinforcement Learning. Proceedings of the 34th International Conference on Machine Learning, in Proceedings of Machine Learning Research 70:459-468 Available from https://proceedings.mlr.press/v70/bello17a.html.

Related Material