Local Group Invariant Representations via Orbit Embeddings

Anant Raj, Abhishek Kumar, Youssef Mroueh, Tom Fletcher, Bernhard Schoelkopf
Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, PMLR 54:1225-1235, 2017.

Abstract

Invariance to nuisance transformations is one of the desirable properties of effective representations. We consider transformations that form a group and propose an approach based on kernel methods to derive local group invariant representations. Locality is achieved by defining a suitable probability distribution over the group which in turn induces distributions in the input feature space. We learn a decision function over these distributions by appealing to the powerful framework of kernel methods and generate local invariant random feature maps via kernel approximations. We show uniform convergence bounds for kernel approximation and provide generalization bounds for learning with these features. We evaluate our method on three real datasets, including Rotated MNIST and CIFAR-10, and observe that it outperforms competing kernel based approaches. The proposed method also outperforms deep CNN on Rotated MNIST and performs comparably to the recently proposed group-equivariant CNN.

Cite this Paper

BibTeX
@InProceedings{pmlr-v54-raj17a, title = {{Local Group Invariant Representations via Orbit Embeddings}}, author = {Raj, Anant and Kumar, Abhishek and Mroueh, Youssef and Fletcher, Tom and Schoelkopf, Bernhard}, booktitle = {Proceedings of the 20th International Conference on Artificial Intelligence and Statistics}, pages = {1225--1235}, year = {2017}, editor = {Singh, Aarti and Zhu, Jerry}, volume = {54}, series = {Proceedings of Machine Learning Research}, month = {20--22 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v54/raj17a/raj17a.pdf}, url = {https://proceedings.mlr.press/v54/raj17a.html}, abstract = {Invariance to nuisance transformations is one of the desirable properties of effective representations. We consider transformations that form a group and propose an approach based on kernel methods to derive local group invariant representations. Locality is achieved by defining a suitable probability distribution over the group which in turn induces distributions in the input feature space. We learn a decision function over these distributions by appealing to the powerful framework of kernel methods and generate local invariant random feature maps via kernel approximations. We show uniform convergence bounds for kernel approximation and provide generalization bounds for learning with these features. We evaluate our method on three real datasets, including Rotated MNIST and CIFAR-10, and observe that it outperforms competing kernel based approaches. The proposed method also outperforms deep CNN on Rotated MNIST and performs comparably to the recently proposed group-equivariant CNN.} }
Endnote
%0 Conference Paper %T Local Group Invariant Representations via Orbit Embeddings %A Anant Raj %A Abhishek Kumar %A Youssef Mroueh %A Tom Fletcher %A Bernhard Schoelkopf %B Proceedings of the 20th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2017 %E Aarti Singh %E Jerry Zhu %F pmlr-v54-raj17a %I PMLR %P 1225--1235 %U https://proceedings.mlr.press/v54/raj17a.html %V 54 %X Invariance to nuisance transformations is one of the desirable properties of effective representations. We consider transformations that form a group and propose an approach based on kernel methods to derive local group invariant representations. Locality is achieved by defining a suitable probability distribution over the group which in turn induces distributions in the input feature space. We learn a decision function over these distributions by appealing to the powerful framework of kernel methods and generate local invariant random feature maps via kernel approximations. We show uniform convergence bounds for kernel approximation and provide generalization bounds for learning with these features. We evaluate our method on three real datasets, including Rotated MNIST and CIFAR-10, and observe that it outperforms competing kernel based approaches. The proposed method also outperforms deep CNN on Rotated MNIST and performs comparably to the recently proposed group-equivariant CNN.
APA
Raj, A., Kumar, A., Mroueh, Y., Fletcher, T. & Schoelkopf, B.. (2017). Local Group Invariant Representations via Orbit Embeddings. Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 54:1225-1235 Available from https://proceedings.mlr.press/v54/raj17a.html.

Related Material