Transportation-Inequalities, Lyapunov Stability and Sampling for Dynamical Systems on Continuous State Space

Muhammad Abdullah Naeem, Miroslav Pajic
Proceedings of The 5th Annual Learning for Dynamics and Control Conference, PMLR 211:653-664, 2023.

Abstract

We study the concentration phenomenon for discrete-time random dynamical systems with an un- bounded state space. We develop a heuristic approach towards obtaining exponential concentration inequalities for dynamical systems using an entirely functional analytic framework. We also show that existence of exponential-type Lyapunov function, compared to the purely deterministic setting, not only implies stability but also exponential concentration inequalities for sampling from the sta- tionary distribution, via transport-entropy inequality (T-E). These results have significant impact in reinforcement learning (RL) and controls, leading to exponential concentration inequalities even for unbounded observables (i.e., rewards), while neither assuming reversibility nor exact knowledge of the considered random dynamical system (assumptions at heart of concentration inequalities in statistical mechanics and Markov diffusion processes).

Cite this Paper

BibTeX
@InProceedings{pmlr-v211-naeem23b, title = {Transportation-Inequalities, Lyapunov Stability and Sampling for Dynamical Systems on Continuous State Space}, author = {Naeem, Muhammad Abdullah and Pajic, Miroslav}, booktitle = {Proceedings of The 5th Annual Learning for Dynamics and Control Conference}, pages = {653--664}, year = {2023}, editor = {Matni, Nikolai and Morari, Manfred and Pappas, George J.}, volume = {211}, series = {Proceedings of Machine Learning Research}, month = {15--16 Jun}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v211/naeem23b/naeem23b.pdf}, url = {https://proceedings.mlr.press/v211/naeem23b.html}, abstract = {We study the concentration phenomenon for discrete-time random dynamical systems with an un- bounded state space. We develop a heuristic approach towards obtaining exponential concentration inequalities for dynamical systems using an entirely functional analytic framework. We also show that existence of exponential-type Lyapunov function, compared to the purely deterministic setting, not only implies stability but also exponential concentration inequalities for sampling from the sta- tionary distribution, via transport-entropy inequality (T-E). These results have significant impact in reinforcement learning (RL) and controls, leading to exponential concentration inequalities even for unbounded observables (i.e., rewards), while neither assuming reversibility nor exact knowledge of the considered random dynamical system (assumptions at heart of concentration inequalities in statistical mechanics and Markov diffusion processes).} }
Endnote
%0 Conference Paper %T Transportation-Inequalities, Lyapunov Stability and Sampling for Dynamical Systems on Continuous State Space %A Muhammad Abdullah Naeem %A Miroslav Pajic %B Proceedings of The 5th Annual Learning for Dynamics and Control Conference %C Proceedings of Machine Learning Research %D 2023 %E Nikolai Matni %E Manfred Morari %E George J. Pappas %F pmlr-v211-naeem23b %I PMLR %P 653--664 %U https://proceedings.mlr.press/v211/naeem23b.html %V 211 %X We study the concentration phenomenon for discrete-time random dynamical systems with an un- bounded state space. We develop a heuristic approach towards obtaining exponential concentration inequalities for dynamical systems using an entirely functional analytic framework. We also show that existence of exponential-type Lyapunov function, compared to the purely deterministic setting, not only implies stability but also exponential concentration inequalities for sampling from the sta- tionary distribution, via transport-entropy inequality (T-E). These results have significant impact in reinforcement learning (RL) and controls, leading to exponential concentration inequalities even for unbounded observables (i.e., rewards), while neither assuming reversibility nor exact knowledge of the considered random dynamical system (assumptions at heart of concentration inequalities in statistical mechanics and Markov diffusion processes).
APA
Naeem, M.A. & Pajic, M.. (2023). Transportation-Inequalities, Lyapunov Stability and Sampling for Dynamical Systems on Continuous State Space. Proceedings of The 5th Annual Learning for Dynamics and Control Conference, in Proceedings of Machine Learning Research 211:653-664 Available from https://proceedings.mlr.press/v211/naeem23b.html.

Related Material