An Adaptive Orthogonal Convolution Scheme for Efficient and Flexible CNN Architectures

Thibaut Boissin, Franck Mamalet, Thomas Fel, Agustin Martin Picard, Thomas Massena, Mathieu Serrurier
Proceedings of the 42nd International Conference on Machine Learning, PMLR 267:4757-4790, 2025.

Abstract

Orthogonal convolutional layers are valuable components in multiple areas of machine learning, such as adversarial robustness, normalizing flows, GANs, and Lipschitz-constrained models. Their ability to preserve norms and ensure stable gradient propagation makes them valuable for a large range of problems. Despite their promise, the deployment of orthogonal convolution in large-scale applications is a significant challenge due to computational overhead and limited support for modern features like strides, dilations, group convolutions, and transposed convolutions. In this paper, we introduce AOC (Adaptive Orthogonal Convolution), a scalable method that extends a previous method (BCOP), effectively overcoming existing limitations in the construction of orthogonal convolutions. This advancement unlocks the construction of architectures that were previously considered impractical. We demonstrate through our experiments that our method produces expressive models that become increasingly efficient as they scale. To foster further advancement, we provide an open-source python package implementing this method, called Orthogonium.

Cite this Paper

BibTeX
@InProceedings{pmlr-v267-boissin25a, title = {An Adaptive Orthogonal Convolution Scheme for Efficient and Flexible {CNN} Architectures}, author = {Boissin, Thibaut and Mamalet, Franck and Fel, Thomas and Picard, Agustin Martin and Massena, Thomas and Serrurier, Mathieu}, booktitle = {Proceedings of the 42nd International Conference on Machine Learning}, pages = {4757--4790}, year = {2025}, editor = {Singh, Aarti and Fazel, Maryam and Hsu, Daniel and Lacoste-Julien, Simon and Berkenkamp, Felix and Maharaj, Tegan and Wagstaff, Kiri and Zhu, Jerry}, volume = {267}, series = {Proceedings of Machine Learning Research}, month = {13--19 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v267/main/assets/boissin25a/boissin25a.pdf}, url = {https://proceedings.mlr.press/v267/boissin25a.html}, abstract = {Orthogonal convolutional layers are valuable components in multiple areas of machine learning, such as adversarial robustness, normalizing flows, GANs, and Lipschitz-constrained models. Their ability to preserve norms and ensure stable gradient propagation makes them valuable for a large range of problems. Despite their promise, the deployment of orthogonal convolution in large-scale applications is a significant challenge due to computational overhead and limited support for modern features like strides, dilations, group convolutions, and transposed convolutions. In this paper, we introduce AOC (Adaptive Orthogonal Convolution), a scalable method that extends a previous method (BCOP), effectively overcoming existing limitations in the construction of orthogonal convolutions. This advancement unlocks the construction of architectures that were previously considered impractical. We demonstrate through our experiments that our method produces expressive models that become increasingly efficient as they scale. To foster further advancement, we provide an open-source python package implementing this method, called Orthogonium.} }
Endnote
%0 Conference Paper %T An Adaptive Orthogonal Convolution Scheme for Efficient and Flexible CNN Architectures %A Thibaut Boissin %A Franck Mamalet %A Thomas Fel %A Agustin Martin Picard %A Thomas Massena %A Mathieu Serrurier %B Proceedings of the 42nd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2025 %E Aarti Singh %E Maryam Fazel %E Daniel Hsu %E Simon Lacoste-Julien %E Felix Berkenkamp %E Tegan Maharaj %E Kiri Wagstaff %E Jerry Zhu %F pmlr-v267-boissin25a %I PMLR %P 4757--4790 %U https://proceedings.mlr.press/v267/boissin25a.html %V 267 %X Orthogonal convolutional layers are valuable components in multiple areas of machine learning, such as adversarial robustness, normalizing flows, GANs, and Lipschitz-constrained models. Their ability to preserve norms and ensure stable gradient propagation makes them valuable for a large range of problems. Despite their promise, the deployment of orthogonal convolution in large-scale applications is a significant challenge due to computational overhead and limited support for modern features like strides, dilations, group convolutions, and transposed convolutions. In this paper, we introduce AOC (Adaptive Orthogonal Convolution), a scalable method that extends a previous method (BCOP), effectively overcoming existing limitations in the construction of orthogonal convolutions. This advancement unlocks the construction of architectures that were previously considered impractical. We demonstrate through our experiments that our method produces expressive models that become increasingly efficient as they scale. To foster further advancement, we provide an open-source python package implementing this method, called Orthogonium.
APA
Boissin, T., Mamalet, F., Fel, T., Picard, A.M., Massena, T. & Serrurier, M.. (2025). An Adaptive Orthogonal Convolution Scheme for Efficient and Flexible CNN Architectures. Proceedings of the 42nd International Conference on Machine Learning, in Proceedings of Machine Learning Research 267:4757-4790 Available from https://proceedings.mlr.press/v267/boissin25a.html.

Related Material