Equivariant Transformer Networks

Kai Sheng Tai, Peter Bailis, Gregory Valiant
Proceedings of the 36th International Conference on Machine Learning, PMLR 97:6086-6095, 2019.

Abstract

How can prior knowledge on the transformation invariances of a domain be incorporated into the architecture of a neural network? We propose Equivariant Transformers (ETs), a family of differentiable image-to-image mappings that improve the robustness of models towards pre-defined continuous transformation groups. Through the use of specially-derived canonical coordinate systems, ETs incorporate functions that are equivariant by construction with respect to these transformations. We show empirically that ETs can be flexibly composed to improve model robustness towards more complicated transformation groups in several parameters. On a real-world image classification task, ETs improve the sample efficiency of ResNet classifiers, achieving relative improvements in error rate of up to 15% in the limited data regime while increasing model parameter count by less than 1%.

Cite this Paper

BibTeX
@InProceedings{pmlr-v97-tai19a, title = {Equivariant Transformer Networks}, author = {Tai, Kai Sheng and Bailis, Peter and Valiant, Gregory}, booktitle = {Proceedings of the 36th International Conference on Machine Learning}, pages = {6086--6095}, year = {2019}, editor = {Chaudhuri, Kamalika and Salakhutdinov, Ruslan}, volume = {97}, series = {Proceedings of Machine Learning Research}, month = {09--15 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v97/tai19a/tai19a.pdf}, url = {https://proceedings.mlr.press/v97/tai19a.html}, abstract = {How can prior knowledge on the transformation invariances of a domain be incorporated into the architecture of a neural network? We propose Equivariant Transformers (ETs), a family of differentiable image-to-image mappings that improve the robustness of models towards pre-defined continuous transformation groups. Through the use of specially-derived canonical coordinate systems, ETs incorporate functions that are equivariant by construction with respect to these transformations. We show empirically that ETs can be flexibly composed to improve model robustness towards more complicated transformation groups in several parameters. On a real-world image classification task, ETs improve the sample efficiency of ResNet classifiers, achieving relative improvements in error rate of up to 15% in the limited data regime while increasing model parameter count by less than 1%.} }
Endnote
%0 Conference Paper %T Equivariant Transformer Networks %A Kai Sheng Tai %A Peter Bailis %A Gregory Valiant %B Proceedings of the 36th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2019 %E Kamalika Chaudhuri %E Ruslan Salakhutdinov %F pmlr-v97-tai19a %I PMLR %P 6086--6095 %U https://proceedings.mlr.press/v97/tai19a.html %V 97 %X How can prior knowledge on the transformation invariances of a domain be incorporated into the architecture of a neural network? We propose Equivariant Transformers (ETs), a family of differentiable image-to-image mappings that improve the robustness of models towards pre-defined continuous transformation groups. Through the use of specially-derived canonical coordinate systems, ETs incorporate functions that are equivariant by construction with respect to these transformations. We show empirically that ETs can be flexibly composed to improve model robustness towards more complicated transformation groups in several parameters. On a real-world image classification task, ETs improve the sample efficiency of ResNet classifiers, achieving relative improvements in error rate of up to 15% in the limited data regime while increasing model parameter count by less than 1%.
APA
Tai, K.S., Bailis, P. & Valiant, G.. (2019). Equivariant Transformer Networks. Proceedings of the 36th International Conference on Machine Learning, in Proceedings of Machine Learning Research 97:6086-6095 Available from https://proceedings.mlr.press/v97/tai19a.html.

Related Material