Equivariance Through Parameter-Sharing

[edit]

Siamak Ravanbakhsh, Jeff Schneider, Barnabás Póczos ;
Proceedings of the 34th International Conference on Machine Learning, PMLR 70:2892-2901, 2017.

Abstract

We propose to study equivariance in deep neural networks through parameter symmetries. In particular, given a group G that acts discretely on the input and output of a standard neural network layer, we show that its equivariance is linked to the symmetry group of network parameters. We then propose two parameter-sharing scheme to induce the desirable symmetry on the parameters of the neural network. Under some conditions on the action of G, our procedure for tying the parameters achieves G-equivariance and guarantees sensitivity to all other permutation groups outside of G.

Related Material