[edit]
Concentration Phenomenon for Random Dynamical Systems: An Operator Theoretic Approach
Proceedings of The 5th Annual Learning for Dynamics and Control Conference, PMLR 211:383-394, 2023.
Abstract
Via operator theoretic methods, we formalize the concentration phenomenon for a given observable ‘r’ of a discrete time Markov chain with ‘μπ’ as invariant ergodic measure, possibly having support on an unbounded state space. The main contribution of this paper is circumventing tedious probabilistic methods with a study of a composition of the Markov transition operator P followed by a multiplication operator defined by er. It turns out that even if the observable/ reward function is unbounded, but for some for some q>2, ‖ and P is hyperbounded with norm control \|P\|_{2 \rightarrow q }< e^{\frac{1}{2}[\frac{1}{2}-\frac{1}{q}]}, sharp non-asymptotic concentration bounds follow. \emph{Transport-entropy} inequality ensures the aforementioned upper bound on multiplication operator for all q>2. The role of \emph{reversibility} in concentration phenomenon is demystified. These results are particularly useful for the reinforcement learning and controls communities as they allow for concentration inequalities w.r.t standard unbounded obersvables/reward functions where exact knowledge of the system is not available, let alone the reversibility of stationary measure.