Population Based Augmentation: Efficient Learning of Augmentation Policy Schedules

Daniel Ho, Eric Liang, Xi Chen, Ion Stoica, Pieter Abbeel
Proceedings of the 36th International Conference on Machine Learning, PMLR 97:2731-2741, 2019.

Abstract

A key challenge in leveraging data augmentation for neural network training is choosing an effective augmentation policy from a large search space of candidate operations. Properly chosen augmentation policies can lead to significant generalization improvements; however, state-of-the-art approaches such as AutoAugment are computationally infeasible to run for the ordinary user. In this paper, we introduce a new data augmentation algorithm, Population Based Augmentation (PBA), which generates nonstationary augmentation policy schedules instead of a fixed augmentation policy. We show that PBA can match the performance of AutoAugment on CIFAR-10, CIFAR-100, and SVHN, with three orders of magnitude less overall compute. On CIFAR-10 we achieve a mean test error of 1.46%, which is a slight improvement upon the current state-of-the-art. The code for PBA is open source and is available at https://github.com/arcelien/pba.

Cite this Paper

BibTeX
@InProceedings{pmlr-v97-ho19b, title = {Population Based Augmentation: Efficient Learning of Augmentation Policy Schedules}, author = {Ho, Daniel and Liang, Eric and Chen, Xi and Stoica, Ion and Abbeel, Pieter}, booktitle = {Proceedings of the 36th International Conference on Machine Learning}, pages = {2731--2741}, year = {2019}, editor = {Chaudhuri, Kamalika and Salakhutdinov, Ruslan}, volume = {97}, series = {Proceedings of Machine Learning Research}, month = {09--15 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v97/ho19b/ho19b.pdf}, url = {https://proceedings.mlr.press/v97/ho19b.html}, abstract = {A key challenge in leveraging data augmentation for neural network training is choosing an effective augmentation policy from a large search space of candidate operations. Properly chosen augmentation policies can lead to significant generalization improvements; however, state-of-the-art approaches such as AutoAugment are computationally infeasible to run for the ordinary user. In this paper, we introduce a new data augmentation algorithm, Population Based Augmentation (PBA), which generates nonstationary augmentation policy schedules instead of a fixed augmentation policy. We show that PBA can match the performance of AutoAugment on CIFAR-10, CIFAR-100, and SVHN, with three orders of magnitude less overall compute. On CIFAR-10 we achieve a mean test error of 1.46%, which is a slight improvement upon the current state-of-the-art. The code for PBA is open source and is available at https://github.com/arcelien/pba.} }
Endnote
%0 Conference Paper %T Population Based Augmentation: Efficient Learning of Augmentation Policy Schedules %A Daniel Ho %A Eric Liang %A Xi Chen %A Ion Stoica %A Pieter Abbeel %B Proceedings of the 36th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2019 %E Kamalika Chaudhuri %E Ruslan Salakhutdinov %F pmlr-v97-ho19b %I PMLR %P 2731--2741 %U https://proceedings.mlr.press/v97/ho19b.html %V 97 %X A key challenge in leveraging data augmentation for neural network training is choosing an effective augmentation policy from a large search space of candidate operations. Properly chosen augmentation policies can lead to significant generalization improvements; however, state-of-the-art approaches such as AutoAugment are computationally infeasible to run for the ordinary user. In this paper, we introduce a new data augmentation algorithm, Population Based Augmentation (PBA), which generates nonstationary augmentation policy schedules instead of a fixed augmentation policy. We show that PBA can match the performance of AutoAugment on CIFAR-10, CIFAR-100, and SVHN, with three orders of magnitude less overall compute. On CIFAR-10 we achieve a mean test error of 1.46%, which is a slight improvement upon the current state-of-the-art. The code for PBA is open source and is available at https://github.com/arcelien/pba.
APA
Ho, D., Liang, E., Chen, X., Stoica, I. & Abbeel, P.. (2019). Population Based Augmentation: Efficient Learning of Augmentation Policy Schedules. Proceedings of the 36th International Conference on Machine Learning, in Proceedings of Machine Learning Research 97:2731-2741 Available from https://proceedings.mlr.press/v97/ho19b.html.

Related Material