Nonlinear Controllability and Function Representation by Neural Stochastic Differential Equations

Tanya Veeravalli, Maxim Raginsky
Proceedings of The 5th Annual Learning for Dynamics and Control Conference, PMLR 211:838-850, 2023.

Abstract

There has been a great deal of recent interest in learning and approximation of functions that can be expressed as expectations of a given nonlinearity with respect to its random internal parameters. Examples of such representations include “infinitely wide” neural nets, where the underlying nonlinearity is given by the activation function of an individual neuron. In this paper, we bring this perspective to function representation by neural stochastic differential equations (SDEs). A neural SDE is an Itô diffusion process whose drift and diffusion matrix are elements of some parametric families. We show that the ability of a neural SDE to realize nonlinear functions of its initial condition can be related to the problem of optimally steering a certain deterministic dynamical system between two given points in finite time. This auxiliary system is obtained by formally replacing the Brownian motion in the SDE by a deterministic control input. We derive upper and lower bounds on the minimum control effort needed to accomplish this steering; these bounds may be of independent interest in the context of motion planning and deterministic optimal control.

Cite this Paper

BibTeX
@InProceedings{pmlr-v211-veeravalli23a, title = {Nonlinear Controllability and Function Representation by Neural Stochastic Differential Equations}, author = {Veeravalli, Tanya and Raginsky, Maxim}, booktitle = {Proceedings of The 5th Annual Learning for Dynamics and Control Conference}, pages = {838--850}, year = {2023}, editor = {Matni, Nikolai and Morari, Manfred and Pappas, George J.}, volume = {211}, series = {Proceedings of Machine Learning Research}, month = {15--16 Jun}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v211/veeravalli23a/veeravalli23a.pdf}, url = {https://proceedings.mlr.press/v211/veeravalli23a.html}, abstract = {There has been a great deal of recent interest in learning and approximation of functions that can be expressed as expectations of a given nonlinearity with respect to its random internal parameters. Examples of such representations include “infinitely wide” neural nets, where the underlying nonlinearity is given by the activation function of an individual neuron. In this paper, we bring this perspective to function representation by neural stochastic differential equations (SDEs). A neural SDE is an Itô diffusion process whose drift and diffusion matrix are elements of some parametric families. We show that the ability of a neural SDE to realize nonlinear functions of its initial condition can be related to the problem of optimally steering a certain deterministic dynamical system between two given points in finite time. This auxiliary system is obtained by formally replacing the Brownian motion in the SDE by a deterministic control input. We derive upper and lower bounds on the minimum control effort needed to accomplish this steering; these bounds may be of independent interest in the context of motion planning and deterministic optimal control.} }
Endnote
%0 Conference Paper %T Nonlinear Controllability and Function Representation by Neural Stochastic Differential Equations %A Tanya Veeravalli %A Maxim Raginsky %B Proceedings of The 5th Annual Learning for Dynamics and Control Conference %C Proceedings of Machine Learning Research %D 2023 %E Nikolai Matni %E Manfred Morari %E George J. Pappas %F pmlr-v211-veeravalli23a %I PMLR %P 838--850 %U https://proceedings.mlr.press/v211/veeravalli23a.html %V 211 %X There has been a great deal of recent interest in learning and approximation of functions that can be expressed as expectations of a given nonlinearity with respect to its random internal parameters. Examples of such representations include “infinitely wide” neural nets, where the underlying nonlinearity is given by the activation function of an individual neuron. In this paper, we bring this perspective to function representation by neural stochastic differential equations (SDEs). A neural SDE is an Itô diffusion process whose drift and diffusion matrix are elements of some parametric families. We show that the ability of a neural SDE to realize nonlinear functions of its initial condition can be related to the problem of optimally steering a certain deterministic dynamical system between two given points in finite time. This auxiliary system is obtained by formally replacing the Brownian motion in the SDE by a deterministic control input. We derive upper and lower bounds on the minimum control effort needed to accomplish this steering; these bounds may be of independent interest in the context of motion planning and deterministic optimal control.
APA
Veeravalli, T. & Raginsky, M.. (2023). Nonlinear Controllability and Function Representation by Neural Stochastic Differential Equations. Proceedings of The 5th Annual Learning for Dynamics and Control Conference, in Proceedings of Machine Learning Research 211:838-850 Available from https://proceedings.mlr.press/v211/veeravalli23a.html.

Related Material