Subhomogeneous Deep Equilibrium Models

Pietro Sittoni, Francesco Tudisco
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:45794-45812, 2024.

Abstract

Implicit-depth neural networks have grown as powerful alternatives to traditional networks in various applications in recent years. However, these models often lack guarantees of existence and uniqueness, raising stability, performance, and reproducibility issues. In this paper, we present a new analysis of the existence and uniqueness of fixed points for implicit-depth neural networks based on the concept of subhomogeneous operators and the nonlinear Perron-Frobenius theory. Compared to previous similar analyses, our theory allows for weaker assumptions on the parameter matrices, thus yielding a more flexible framework for well-defined implicit networks. We illustrate the performance of the resulting subhomogeneous networks on feedforward, convolutional, and graph neural network examples

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-sittoni24a, title = {Subhomogeneous Deep Equilibrium Models}, author = {Sittoni, Pietro and Tudisco, Francesco}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {45794--45812}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/sittoni24a/sittoni24a.pdf}, url = {https://proceedings.mlr.press/v235/sittoni24a.html}, abstract = {Implicit-depth neural networks have grown as powerful alternatives to traditional networks in various applications in recent years. However, these models often lack guarantees of existence and uniqueness, raising stability, performance, and reproducibility issues. In this paper, we present a new analysis of the existence and uniqueness of fixed points for implicit-depth neural networks based on the concept of subhomogeneous operators and the nonlinear Perron-Frobenius theory. Compared to previous similar analyses, our theory allows for weaker assumptions on the parameter matrices, thus yielding a more flexible framework for well-defined implicit networks. We illustrate the performance of the resulting subhomogeneous networks on feedforward, convolutional, and graph neural network examples} }
Endnote
%0 Conference Paper %T Subhomogeneous Deep Equilibrium Models %A Pietro Sittoni %A Francesco Tudisco %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-sittoni24a %I PMLR %P 45794--45812 %U https://proceedings.mlr.press/v235/sittoni24a.html %V 235 %X Implicit-depth neural networks have grown as powerful alternatives to traditional networks in various applications in recent years. However, these models often lack guarantees of existence and uniqueness, raising stability, performance, and reproducibility issues. In this paper, we present a new analysis of the existence and uniqueness of fixed points for implicit-depth neural networks based on the concept of subhomogeneous operators and the nonlinear Perron-Frobenius theory. Compared to previous similar analyses, our theory allows for weaker assumptions on the parameter matrices, thus yielding a more flexible framework for well-defined implicit networks. We illustrate the performance of the resulting subhomogeneous networks on feedforward, convolutional, and graph neural network examples
APA
Sittoni, P. & Tudisco, F.. (2024). Subhomogeneous Deep Equilibrium Models. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:45794-45812 Available from https://proceedings.mlr.press/v235/sittoni24a.html.

Related Material