Equivariant Frames and the Impossibility of Continuous Canonicalization

Nadav Dym, Hannah Lawrence, Jonathan W. Siegel
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:12228-12267, 2024.

Abstract

Canonicalization provides an architecture-agnostic method for enforcing equivariance, with generalizations such as frame-averaging recently gaining prominence as a lightweight and flexible alternative to equivariant architectures. Recent works have found an empirical benefit to using probabilistic frames instead, which learn weighted distributions over group elements. In this work, we provide strong theoretical justification for this phenomenon: for commonly-used groups, there is no efficiently computable choice of frame that preserves continuity of the function being averaged. In other words, unweighted frame-averaging can turn a smooth, non-symmetric function into a discontinuous, symmetric function. To address this fundamental robustness problem, we formally define and construct weighted frames, which provably preserve continuity, and demonstrate their utility by constructing efficient and continuous weighted frames for the actions of $SO(d)$, $O(d)$, and $S_n$ on point clouds.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-dym24a, title = {Equivariant Frames and the Impossibility of Continuous Canonicalization}, author = {Dym, Nadav and Lawrence, Hannah and Siegel, Jonathan W.}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {12228--12267}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/dym24a/dym24a.pdf}, url = {https://proceedings.mlr.press/v235/dym24a.html}, abstract = {Canonicalization provides an architecture-agnostic method for enforcing equivariance, with generalizations such as frame-averaging recently gaining prominence as a lightweight and flexible alternative to equivariant architectures. Recent works have found an empirical benefit to using probabilistic frames instead, which learn weighted distributions over group elements. In this work, we provide strong theoretical justification for this phenomenon: for commonly-used groups, there is no efficiently computable choice of frame that preserves continuity of the function being averaged. In other words, unweighted frame-averaging can turn a smooth, non-symmetric function into a discontinuous, symmetric function. To address this fundamental robustness problem, we formally define and construct weighted frames, which provably preserve continuity, and demonstrate their utility by constructing efficient and continuous weighted frames for the actions of $SO(d)$, $O(d)$, and $S_n$ on point clouds.} }
Endnote
%0 Conference Paper %T Equivariant Frames and the Impossibility of Continuous Canonicalization %A Nadav Dym %A Hannah Lawrence %A Jonathan W. Siegel %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-dym24a %I PMLR %P 12228--12267 %U https://proceedings.mlr.press/v235/dym24a.html %V 235 %X Canonicalization provides an architecture-agnostic method for enforcing equivariance, with generalizations such as frame-averaging recently gaining prominence as a lightweight and flexible alternative to equivariant architectures. Recent works have found an empirical benefit to using probabilistic frames instead, which learn weighted distributions over group elements. In this work, we provide strong theoretical justification for this phenomenon: for commonly-used groups, there is no efficiently computable choice of frame that preserves continuity of the function being averaged. In other words, unweighted frame-averaging can turn a smooth, non-symmetric function into a discontinuous, symmetric function. To address this fundamental robustness problem, we formally define and construct weighted frames, which provably preserve continuity, and demonstrate their utility by constructing efficient and continuous weighted frames for the actions of $SO(d)$, $O(d)$, and $S_n$ on point clouds.
APA
Dym, N., Lawrence, H. & Siegel, J.W.. (2024). Equivariant Frames and the Impossibility of Continuous Canonicalization. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:12228-12267 Available from https://proceedings.mlr.press/v235/dym24a.html.

Related Material