Learning Instance-Specific Augmentations by Capturing Local Invariances

Ning Miao, Tom Rainforth, Emile Mathieu, Yann Dubois, Yee Whye Teh, Adam Foster, Hyunjik Kim
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:24720-24736, 2023.

Abstract

We introduce InstaAug, a method for automatically learning input-specific augmentations from data. Previous methods for learning augmentations have typically assumed independence between the original input and the transformation applied to that input. This can be highly restrictive, as the invariances we hope our augmentation will capture are themselves often highly input dependent. InstaAug instead introduces a learnable invariance module that maps from inputs to tailored transformation parameters, allowing local invariances to be captured. This can be simultaneously trained alongside the downstream model in a fully end-to-end manner, or separately learned for a pre-trained model. We empirically demonstrate that InstaAug learns meaningful input-dependent augmentations for a wide range of transformation classes, which in turn provides better performance on both supervised and self-supervised tasks.

Cite this Paper

BibTeX
@InProceedings{pmlr-v202-miao23a, title = {Learning Instance-Specific Augmentations by Capturing Local Invariances}, author = {Miao, Ning and Rainforth, Tom and Mathieu, Emile and Dubois, Yann and Teh, Yee Whye and Foster, Adam and Kim, Hyunjik}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {24720--24736}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/miao23a/miao23a.pdf}, url = {https://proceedings.mlr.press/v202/miao23a.html}, abstract = {We introduce InstaAug, a method for automatically learning input-specific augmentations from data. Previous methods for learning augmentations have typically assumed independence between the original input and the transformation applied to that input. This can be highly restrictive, as the invariances we hope our augmentation will capture are themselves often highly input dependent. InstaAug instead introduces a learnable invariance module that maps from inputs to tailored transformation parameters, allowing local invariances to be captured. This can be simultaneously trained alongside the downstream model in a fully end-to-end manner, or separately learned for a pre-trained model. We empirically demonstrate that InstaAug learns meaningful input-dependent augmentations for a wide range of transformation classes, which in turn provides better performance on both supervised and self-supervised tasks.} }
Endnote
%0 Conference Paper %T Learning Instance-Specific Augmentations by Capturing Local Invariances %A Ning Miao %A Tom Rainforth %A Emile Mathieu %A Yann Dubois %A Yee Whye Teh %A Adam Foster %A Hyunjik Kim %B Proceedings of the 40th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Andreas Krause %E Emma Brunskill %E Kyunghyun Cho %E Barbara Engelhardt %E Sivan Sabato %E Jonathan Scarlett %F pmlr-v202-miao23a %I PMLR %P 24720--24736 %U https://proceedings.mlr.press/v202/miao23a.html %V 202 %X We introduce InstaAug, a method for automatically learning input-specific augmentations from data. Previous methods for learning augmentations have typically assumed independence between the original input and the transformation applied to that input. This can be highly restrictive, as the invariances we hope our augmentation will capture are themselves often highly input dependent. InstaAug instead introduces a learnable invariance module that maps from inputs to tailored transformation parameters, allowing local invariances to be captured. This can be simultaneously trained alongside the downstream model in a fully end-to-end manner, or separately learned for a pre-trained model. We empirically demonstrate that InstaAug learns meaningful input-dependent augmentations for a wide range of transformation classes, which in turn provides better performance on both supervised and self-supervised tasks.
APA
Miao, N., Rainforth, T., Mathieu, E., Dubois, Y., Teh, Y.W., Foster, A. & Kim, H.. (2023). Learning Instance-Specific Augmentations by Capturing Local Invariances. Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:24720-24736 Available from https://proceedings.mlr.press/v202/miao23a.html.

Related Material