Learning Kolmogorov-Arnold Neural Activation Functions by Infinite-Dimensional Optimization

Leon Khalyavin, Alessio Moreschini, Thomas Parisini
Proceedings of the 7th Annual Learning for Dynamics \& Control Conference, PMLR 283:443-455, 2025.

Abstract

Inspired by the Kolmogorov-Arnold Representation Theorem, Kolmogorov-Arnold Networks (KAN) have recently reshaped the landscape of functional representation in the context of training univariate activation functions, thus achieving remarkable performance gains over traditional Multi-Layer Perceptron (MLPs). However, the parametrization of the activation functions in KANs hinders their scalability and usage in machine-learning applications. In this article, we propose a novel infinite-dimensional optimization framework to learn the activation functions. Our game-changing approach enables the achievement of boundedness, interpretability, and, seamless compatibility with training by backpropagation are all preserved for the network, while also significantly reducing the number of parameters required to represent each edge. Through functional representation examples, our results reveal superior accuracy in tasks such as functional representation, positioning our method as a transformative leap forward in neural network design and optimization.

Cite this Paper

BibTeX
@InProceedings{pmlr-v283-khalyavin25a, title = {Learning Kolmogorov-Arnold Neural Activation Functions by Infinite-Dimensional Optimization}, author = {Khalyavin, Leon and Moreschini, Alessio and Parisini, Thomas}, booktitle = {Proceedings of the 7th Annual Learning for Dynamics \& Control Conference}, pages = {443--455}, year = {2025}, editor = {Ozay, Necmiye and Balzano, Laura and Panagou, Dimitra and Abate, Alessandro}, volume = {283}, series = {Proceedings of Machine Learning Research}, month = {04--06 Jun}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v283/main/assets/khalyavin25a/khalyavin25a.pdf}, url = {https://proceedings.mlr.press/v283/khalyavin25a.html}, abstract = {Inspired by the Kolmogorov-Arnold Representation Theorem, Kolmogorov-Arnold Networks (KAN) have recently reshaped the landscape of functional representation in the context of training univariate activation functions, thus achieving remarkable performance gains over traditional Multi-Layer Perceptron (MLPs). However, the parametrization of the activation functions in KANs hinders their scalability and usage in machine-learning applications. In this article, we propose a novel infinite-dimensional optimization framework to learn the activation functions. Our game-changing approach enables the achievement of boundedness, interpretability, and, seamless compatibility with training by backpropagation are all preserved for the network, while also significantly reducing the number of parameters required to represent each edge. Through functional representation examples, our results reveal superior accuracy in tasks such as functional representation, positioning our method as a transformative leap forward in neural network design and optimization.} }
Endnote
%0 Conference Paper %T Learning Kolmogorov-Arnold Neural Activation Functions by Infinite-Dimensional Optimization %A Leon Khalyavin %A Alessio Moreschini %A Thomas Parisini %B Proceedings of the 7th Annual Learning for Dynamics \& Control Conference %C Proceedings of Machine Learning Research %D 2025 %E Necmiye Ozay %E Laura Balzano %E Dimitra Panagou %E Alessandro Abate %F pmlr-v283-khalyavin25a %I PMLR %P 443--455 %U https://proceedings.mlr.press/v283/khalyavin25a.html %V 283 %X Inspired by the Kolmogorov-Arnold Representation Theorem, Kolmogorov-Arnold Networks (KAN) have recently reshaped the landscape of functional representation in the context of training univariate activation functions, thus achieving remarkable performance gains over traditional Multi-Layer Perceptron (MLPs). However, the parametrization of the activation functions in KANs hinders their scalability and usage in machine-learning applications. In this article, we propose a novel infinite-dimensional optimization framework to learn the activation functions. Our game-changing approach enables the achievement of boundedness, interpretability, and, seamless compatibility with training by backpropagation are all preserved for the network, while also significantly reducing the number of parameters required to represent each edge. Through functional representation examples, our results reveal superior accuracy in tasks such as functional representation, positioning our method as a transformative leap forward in neural network design and optimization.
APA
Khalyavin, L., Moreschini, A. & Parisini, T.. (2025). Learning Kolmogorov-Arnold Neural Activation Functions by Infinite-Dimensional Optimization. Proceedings of the 7th Annual Learning for Dynamics \& Control Conference, in Proceedings of Machine Learning Research 283:443-455 Available from https://proceedings.mlr.press/v283/khalyavin25a.html.

Related Material