A Probabilistic Approach to Learning the Degree of Equivariance in Steerable CNNs

Lars Veefkind, Gabriele Cesa
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:49249-49309, 2024.

Abstract

Steerable convolutional neural networks (SCNNs) enhance task performance by modelling geometric symmetries through equivariance constraints on weights. Yet, unknown or varying symmetries can lead to overconstrained weights and decreased performance. To address this, this paper introduces a probabilistic method to learn the degree of equivariance in SCNNs. We parameterise the degree of equivariance as a likelihood distribution over the transformation group using Fourier coefficients, offering the option to model layer-wise and shared equivariance. These likelihood distributions are regularised to ensure an interpretable degree of equivariance across the network. Advantages include the applicability to many types of equivariant networks through the flexible framework of SCNNs and the ability to learn equivariance with respect to any subgroup of any compact group without requiring additional layers. Our experiments reveal competitive performance on datasets with mixed symmetries, with learnt likelihood distributions that are representative of the underlying degree of equivariance.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-veefkind24a, title = {A Probabilistic Approach to Learning the Degree of Equivariance in Steerable {CNN}s}, author = {Veefkind, Lars and Cesa, Gabriele}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {49249--49309}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/veefkind24a/veefkind24a.pdf}, url = {https://proceedings.mlr.press/v235/veefkind24a.html}, abstract = {Steerable convolutional neural networks (SCNNs) enhance task performance by modelling geometric symmetries through equivariance constraints on weights. Yet, unknown or varying symmetries can lead to overconstrained weights and decreased performance. To address this, this paper introduces a probabilistic method to learn the degree of equivariance in SCNNs. We parameterise the degree of equivariance as a likelihood distribution over the transformation group using Fourier coefficients, offering the option to model layer-wise and shared equivariance. These likelihood distributions are regularised to ensure an interpretable degree of equivariance across the network. Advantages include the applicability to many types of equivariant networks through the flexible framework of SCNNs and the ability to learn equivariance with respect to any subgroup of any compact group without requiring additional layers. Our experiments reveal competitive performance on datasets with mixed symmetries, with learnt likelihood distributions that are representative of the underlying degree of equivariance.} }
Endnote
%0 Conference Paper %T A Probabilistic Approach to Learning the Degree of Equivariance in Steerable CNNs %A Lars Veefkind %A Gabriele Cesa %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-veefkind24a %I PMLR %P 49249--49309 %U https://proceedings.mlr.press/v235/veefkind24a.html %V 235 %X Steerable convolutional neural networks (SCNNs) enhance task performance by modelling geometric symmetries through equivariance constraints on weights. Yet, unknown or varying symmetries can lead to overconstrained weights and decreased performance. To address this, this paper introduces a probabilistic method to learn the degree of equivariance in SCNNs. We parameterise the degree of equivariance as a likelihood distribution over the transformation group using Fourier coefficients, offering the option to model layer-wise and shared equivariance. These likelihood distributions are regularised to ensure an interpretable degree of equivariance across the network. Advantages include the applicability to many types of equivariant networks through the flexible framework of SCNNs and the ability to learn equivariance with respect to any subgroup of any compact group without requiring additional layers. Our experiments reveal competitive performance on datasets with mixed symmetries, with learnt likelihood distributions that are representative of the underlying degree of equivariance.
APA
Veefkind, L. & Cesa, G.. (2024). A Probabilistic Approach to Learning the Degree of Equivariance in Steerable CNNs. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:49249-49309 Available from https://proceedings.mlr.press/v235/veefkind24a.html.

Related Material