Learning on Manifolds: Universal Approximations Properties using Geometric Controllability Conditions for Neural ODEs

Karthik Elamvazhuthi, Xuechen Zhang, Samet Oymak, Fabio Pasqualetti
Proceedings of The 5th Annual Learning for Dynamics and Control Conference, PMLR 211:1-11, 2023.

Abstract

In numerous robotics and mechanical engineering applications, among others, data is often constrained on smooth manifolds due to the presence of rotational degrees of freedom. Common data-driven and learning-based methods such as neural ordinary differential equations (ODEs), however, typically fail to satisfy these manifold constraints and perform poorly for these applications. To address this shortcoming, in this paper we study a class of neural ordinary differential equations that, by design, leave a given manifold invariant, and characterize their properties by leveraging the controllability properties of control affine systems. In particular, using a result due to Agrachev and Caponigro on approximating diffeomorphisms with flows of feedback control systems, we show that any map that can be represented as the flow of a manifold-constrained dynamical system can also be approximated using the flow of manifold-constrained neural ODE, whenever a certain controllability condition is satisfied. Additionally, we show that this universal approximation property holds when the neural ODE has limited width in each layer, thus leveraging the depth of network instead for approximation. We verify our theoretical findings using numerical experiments on PyTorch for the manifolds $S^2$ and the 3-dimensional orthogonal group $SO(3)$, which are model manifolds for mechanical systems such as spacecrafts and satellites. We also compare the performance of the manifold invariant neural ODE with classical neural ODEs that ignore the manifold invariant properties and show the superiority of our approach in terms of accuracy and sample complexity.

Cite this Paper

BibTeX
@InProceedings{pmlr-v211-elamvazhuthi23a, title = {Learning on Manifolds: Universal Approximations Properties using Geometric Controllability Conditions for Neural ODEs}, author = {Elamvazhuthi, Karthik and Zhang, Xuechen and Oymak, Samet and Pasqualetti, Fabio}, booktitle = {Proceedings of The 5th Annual Learning for Dynamics and Control Conference}, pages = {1--11}, year = {2023}, editor = {Matni, Nikolai and Morari, Manfred and Pappas, George J.}, volume = {211}, series = {Proceedings of Machine Learning Research}, month = {15--16 Jun}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v211/elamvazhuthi23a/elamvazhuthi23a.pdf}, url = {https://proceedings.mlr.press/v211/elamvazhuthi23a.html}, abstract = {In numerous robotics and mechanical engineering applications, among others, data is often constrained on smooth manifolds due to the presence of rotational degrees of freedom. Common data-driven and learning-based methods such as neural ordinary differential equations (ODEs), however, typically fail to satisfy these manifold constraints and perform poorly for these applications. To address this shortcoming, in this paper we study a class of neural ordinary differential equations that, by design, leave a given manifold invariant, and characterize their properties by leveraging the controllability properties of control affine systems. In particular, using a result due to Agrachev and Caponigro on approximating diffeomorphisms with flows of feedback control systems, we show that any map that can be represented as the flow of a manifold-constrained dynamical system can also be approximated using the flow of manifold-constrained neural ODE, whenever a certain controllability condition is satisfied. Additionally, we show that this universal approximation property holds when the neural ODE has limited width in each layer, thus leveraging the depth of network instead for approximation. We verify our theoretical findings using numerical experiments on PyTorch for the manifolds $S^2$ and the 3-dimensional orthogonal group $SO(3)$, which are model manifolds for mechanical systems such as spacecrafts and satellites. We also compare the performance of the manifold invariant neural ODE with classical neural ODEs that ignore the manifold invariant properties and show the superiority of our approach in terms of accuracy and sample complexity. } }
Endnote
%0 Conference Paper %T Learning on Manifolds: Universal Approximations Properties using Geometric Controllability Conditions for Neural ODEs %A Karthik Elamvazhuthi %A Xuechen Zhang %A Samet Oymak %A Fabio Pasqualetti %B Proceedings of The 5th Annual Learning for Dynamics and Control Conference %C Proceedings of Machine Learning Research %D 2023 %E Nikolai Matni %E Manfred Morari %E George J. Pappas %F pmlr-v211-elamvazhuthi23a %I PMLR %P 1--11 %U https://proceedings.mlr.press/v211/elamvazhuthi23a.html %V 211 %X In numerous robotics and mechanical engineering applications, among others, data is often constrained on smooth manifolds due to the presence of rotational degrees of freedom. Common data-driven and learning-based methods such as neural ordinary differential equations (ODEs), however, typically fail to satisfy these manifold constraints and perform poorly for these applications. To address this shortcoming, in this paper we study a class of neural ordinary differential equations that, by design, leave a given manifold invariant, and characterize their properties by leveraging the controllability properties of control affine systems. In particular, using a result due to Agrachev and Caponigro on approximating diffeomorphisms with flows of feedback control systems, we show that any map that can be represented as the flow of a manifold-constrained dynamical system can also be approximated using the flow of manifold-constrained neural ODE, whenever a certain controllability condition is satisfied. Additionally, we show that this universal approximation property holds when the neural ODE has limited width in each layer, thus leveraging the depth of network instead for approximation. We verify our theoretical findings using numerical experiments on PyTorch for the manifolds $S^2$ and the 3-dimensional orthogonal group $SO(3)$, which are model manifolds for mechanical systems such as spacecrafts and satellites. We also compare the performance of the manifold invariant neural ODE with classical neural ODEs that ignore the manifold invariant properties and show the superiority of our approach in terms of accuracy and sample complexity.
APA
Elamvazhuthi, K., Zhang, X., Oymak, S. & Pasqualetti, F.. (2023). Learning on Manifolds: Universal Approximations Properties using Geometric Controllability Conditions for Neural ODEs. Proceedings of The 5th Annual Learning for Dynamics and Control Conference, in Proceedings of Machine Learning Research 211:1-11 Available from https://proceedings.mlr.press/v211/elamvazhuthi23a.html.

Related Material