Hyperbolic Optimizer as a Dynamical System

Nico Alvarado, Hans Lobel
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:1243-1260, 2024.

Abstract

During the last few years, the field of dynamical systems has been developing innovative tools to study the asymptotic behavior of different optimizers in the context of neural networks. In this work, we redefine an extensively studied optimizer, employing classical techniques from hyperbolic geometry. This new definition is linked to a non-linear differential equation as a continuous limit. Additionally, by utilizing Lyapunov stability concepts, we analyze the asymptotic behavior of its critical points.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-alvarado24a, title = {Hyperbolic Optimizer as a Dynamical System}, author = {Alvarado, Nico and Lobel, Hans}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {1243--1260}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/alvarado24a/alvarado24a.pdf}, url = {https://proceedings.mlr.press/v235/alvarado24a.html}, abstract = {During the last few years, the field of dynamical systems has been developing innovative tools to study the asymptotic behavior of different optimizers in the context of neural networks. In this work, we redefine an extensively studied optimizer, employing classical techniques from hyperbolic geometry. This new definition is linked to a non-linear differential equation as a continuous limit. Additionally, by utilizing Lyapunov stability concepts, we analyze the asymptotic behavior of its critical points.} }
Endnote
%0 Conference Paper %T Hyperbolic Optimizer as a Dynamical System %A Nico Alvarado %A Hans Lobel %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-alvarado24a %I PMLR %P 1243--1260 %U https://proceedings.mlr.press/v235/alvarado24a.html %V 235 %X During the last few years, the field of dynamical systems has been developing innovative tools to study the asymptotic behavior of different optimizers in the context of neural networks. In this work, we redefine an extensively studied optimizer, employing classical techniques from hyperbolic geometry. This new definition is linked to a non-linear differential equation as a continuous limit. Additionally, by utilizing Lyapunov stability concepts, we analyze the asymptotic behavior of its critical points.
APA
Alvarado, N. & Lobel, H.. (2024). Hyperbolic Optimizer as a Dynamical System. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:1243-1260 Available from https://proceedings.mlr.press/v235/alvarado24a.html.

Related Material