On the Geometry and Optimization of Polynomial Convolutional Networks

Vahid Shahverdi, Giovanni Luca Marchetti, Kathlén Kohn
Proceedings of The 28th International Conference on Artificial Intelligence and Statistics, PMLR 258:604-612, 2025.

Abstract

We study convolutional neural networks with monomial activation functions. Specifically, we prove that their parameterization map is regular and is an isomorphism almost everywhere, up to rescaling the filters. By leveraging on tools from algebraic geometry, we explore the geometric properties of the image in function space of this map – typically referred to as neuromanifold. In particular, we compute the dimension and the degree of the neuromanifold, which measure the expressivity of the model, and describe its singularities. Moreover, for a generic large dataset, we derive an explicit formula that quantifies the number of critical points arising in the optimization of a regression loss.

Cite this Paper

BibTeX
@InProceedings{pmlr-v258-shahverdi25a, title = {On the Geometry and Optimization of Polynomial Convolutional Networks}, author = {Shahverdi, Vahid and Marchetti, Giovanni Luca and Kohn, Kathl{\'e}n}, booktitle = {Proceedings of The 28th International Conference on Artificial Intelligence and Statistics}, pages = {604--612}, year = {2025}, editor = {Li, Yingzhen and Mandt, Stephan and Agrawal, Shipra and Khan, Emtiyaz}, volume = {258}, series = {Proceedings of Machine Learning Research}, month = {03--05 May}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v258/main/assets/shahverdi25a/shahverdi25a.pdf}, url = {https://proceedings.mlr.press/v258/shahverdi25a.html}, abstract = {We study convolutional neural networks with monomial activation functions. Specifically, we prove that their parameterization map is regular and is an isomorphism almost everywhere, up to rescaling the filters. By leveraging on tools from algebraic geometry, we explore the geometric properties of the image in function space of this map – typically referred to as neuromanifold. In particular, we compute the dimension and the degree of the neuromanifold, which measure the expressivity of the model, and describe its singularities. Moreover, for a generic large dataset, we derive an explicit formula that quantifies the number of critical points arising in the optimization of a regression loss.} }
Endnote
%0 Conference Paper %T On the Geometry and Optimization of Polynomial Convolutional Networks %A Vahid Shahverdi %A Giovanni Luca Marchetti %A Kathlén Kohn %B Proceedings of The 28th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2025 %E Yingzhen Li %E Stephan Mandt %E Shipra Agrawal %E Emtiyaz Khan %F pmlr-v258-shahverdi25a %I PMLR %P 604--612 %U https://proceedings.mlr.press/v258/shahverdi25a.html %V 258 %X We study convolutional neural networks with monomial activation functions. Specifically, we prove that their parameterization map is regular and is an isomorphism almost everywhere, up to rescaling the filters. By leveraging on tools from algebraic geometry, we explore the geometric properties of the image in function space of this map – typically referred to as neuromanifold. In particular, we compute the dimension and the degree of the neuromanifold, which measure the expressivity of the model, and describe its singularities. Moreover, for a generic large dataset, we derive an explicit formula that quantifies the number of critical points arising in the optimization of a regression loss.
APA
Shahverdi, V., Marchetti, G.L. & Kohn, K.. (2025). On the Geometry and Optimization of Polynomial Convolutional Networks. Proceedings of The 28th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 258:604-612 Available from https://proceedings.mlr.press/v258/shahverdi25a.html.

Related Material