Universal Equivariant Multilayer Perceptrons

Siamak Ravanbakhsh
Proceedings of the 37th International Conference on Machine Learning, PMLR 119:7996-8006, 2020.

Abstract

Group invariant and equivariant Multilayer Perceptrons (MLP), also known as Equivariant Networks and Group Group Convolutional Neural Networks (G-CNN) have achieved remarkable success in learning on a variety of data structures, such as sequences, images, sets, and graphs. This paper proves the universality of a broad class of equivariant MLPs with a single hidden layer. In particular, it is shown that having a hidden layer on which the group acts regularly is sufficient for universal equivariance (invariance). For example, some types of steerable-CNN’s become universal. Another corollary is the unconditional universality of equivariant MLPs for all Abelian groups. A third corollary is the universality of equivariant MLPs with a high-order hidden layer, where we give both group-agnostic bounds and group-specific bounds on the order of the hidden layer that guarantees universal equivariance.

Cite this Paper

BibTeX
@InProceedings{pmlr-v119-ravanbakhsh20a, title = {Universal Equivariant Multilayer Perceptrons}, author = {Ravanbakhsh, Siamak}, booktitle = {Proceedings of the 37th International Conference on Machine Learning}, pages = {7996--8006}, year = {2020}, editor = {III, Hal Daumé and Singh, Aarti}, volume = {119}, series = {Proceedings of Machine Learning Research}, month = {13--18 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v119/ravanbakhsh20a/ravanbakhsh20a.pdf}, url = {https://proceedings.mlr.press/v119/ravanbakhsh20a.html}, abstract = {Group invariant and equivariant Multilayer Perceptrons (MLP), also known as Equivariant Networks and Group Group Convolutional Neural Networks (G-CNN) have achieved remarkable success in learning on a variety of data structures, such as sequences, images, sets, and graphs. This paper proves the universality of a broad class of equivariant MLPs with a single hidden layer. In particular, it is shown that having a hidden layer on which the group acts regularly is sufficient for universal equivariance (invariance). For example, some types of steerable-CNN’s become universal. Another corollary is the unconditional universality of equivariant MLPs for all Abelian groups. A third corollary is the universality of equivariant MLPs with a high-order hidden layer, where we give both group-agnostic bounds and group-specific bounds on the order of the hidden layer that guarantees universal equivariance.} }
Endnote
%0 Conference Paper %T Universal Equivariant Multilayer Perceptrons %A Siamak Ravanbakhsh %B Proceedings of the 37th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2020 %E Hal Daumé III %E Aarti Singh %F pmlr-v119-ravanbakhsh20a %I PMLR %P 7996--8006 %U https://proceedings.mlr.press/v119/ravanbakhsh20a.html %V 119 %X Group invariant and equivariant Multilayer Perceptrons (MLP), also known as Equivariant Networks and Group Group Convolutional Neural Networks (G-CNN) have achieved remarkable success in learning on a variety of data structures, such as sequences, images, sets, and graphs. This paper proves the universality of a broad class of equivariant MLPs with a single hidden layer. In particular, it is shown that having a hidden layer on which the group acts regularly is sufficient for universal equivariance (invariance). For example, some types of steerable-CNN’s become universal. Another corollary is the unconditional universality of equivariant MLPs for all Abelian groups. A third corollary is the universality of equivariant MLPs with a high-order hidden layer, where we give both group-agnostic bounds and group-specific bounds on the order of the hidden layer that guarantees universal equivariance.
APA
Ravanbakhsh, S.. (2020). Universal Equivariant Multilayer Perceptrons. Proceedings of the 37th International Conference on Machine Learning, in Proceedings of Machine Learning Research 119:7996-8006 Available from https://proceedings.mlr.press/v119/ravanbakhsh20a.html.

Related Material