Unifying transformers and convolutional networks as equivariant maps

Elias Nyholm
Proceedings of the Geometry, Topology, and Machine Learning Workshop, PMLR 325:240-245, 2026.

Abstract

Motivated by the prevalence of equivariant machine learning models and the success of the framework of linear equivariant convolutional neural networks, we present in this work an extended framework that also includes non-linear equivariant models. More specifically, we represent these models as integral operators and derive conditions on the integrand for the operator to be equivariant. Further, we prove the generality of the proposed framework and show explicitly how common equivariant models, linear as well as non-linear, fit into the proposed formulation. This extended abstract summarises the central points of the preprint \citet{nyholm2025equivariantnonlinearmapsneural}, which is joint work together with Oscar Carlsson, Maurice Weiler and Daniel Persson.

Cite this Paper

BibTeX
@InProceedings{pmlr-v325-nyholm26a, title = {Unifying transformers and convolutional networks as equivariant maps}, author = {Nyholm, Elias}, booktitle = {Proceedings of the Geometry, Topology, and Machine Learning Workshop}, pages = {240--245}, year = {2026}, editor = {Bleher, Michael and Jensen, Freya and Maier, Levin and Taha, Diaaeldin and Wienhard, Anna}, volume = {325}, series = {Proceedings of Machine Learning Research}, month = {10--14 Nov}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v325/main/assets/nyholm26a/nyholm26a.pdf}, url = {https://proceedings.mlr.press/v325/nyholm26a.html}, abstract = {Motivated by the prevalence of equivariant machine learning models and the success of the framework of linear equivariant convolutional neural networks, we present in this work an extended framework that also includes non-linear equivariant models. More specifically, we represent these models as integral operators and derive conditions on the integrand for the operator to be equivariant. Further, we prove the generality of the proposed framework and show explicitly how common equivariant models, linear as well as non-linear, fit into the proposed formulation. This extended abstract summarises the central points of the preprint \citet{nyholm2025equivariantnonlinearmapsneural}, which is joint work together with Oscar Carlsson, Maurice Weiler and Daniel Persson.} }
Endnote
%0 Conference Paper %T Unifying transformers and convolutional networks as equivariant maps %A Elias Nyholm %B Proceedings of the Geometry, Topology, and Machine Learning Workshop %C Proceedings of Machine Learning Research %D 2026 %E Michael Bleher %E Freya Jensen %E Levin Maier %E Diaaeldin Taha %E Anna Wienhard %F pmlr-v325-nyholm26a %I PMLR %P 240--245 %U https://proceedings.mlr.press/v325/nyholm26a.html %V 325 %X Motivated by the prevalence of equivariant machine learning models and the success of the framework of linear equivariant convolutional neural networks, we present in this work an extended framework that also includes non-linear equivariant models. More specifically, we represent these models as integral operators and derive conditions on the integrand for the operator to be equivariant. Further, we prove the generality of the proposed framework and show explicitly how common equivariant models, linear as well as non-linear, fit into the proposed formulation. This extended abstract summarises the central points of the preprint \citet{nyholm2025equivariantnonlinearmapsneural}, which is joint work together with Oscar Carlsson, Maurice Weiler and Daniel Persson.
APA
Nyholm, E.. (2026). Unifying transformers and convolutional networks as equivariant maps. Proceedings of the Geometry, Topology, and Machine Learning Workshop, in Proceedings of Machine Learning Research 325:240-245 Available from https://proceedings.mlr.press/v325/nyholm26a.html.

Related Material