Interacting Particle Systems for Fast Linear Quadratic RL

Anant A. Joshi, Heng-Sheng Chang, Amirhossein Taghvaei, Prashant G. Mehta, Sean P. Meyn
Proceedings of the 7th Annual Learning for Dynamics \& Control Conference, PMLR 283:99-111, 2025.

Abstract

This paper is concerned with the design of algorithms based on systems of interacting particles to represent, approximate, and learn the optimal control law for reinforcement learning (RL). The primary contribution is that convergence rates are greatly accelerated by the interactions between particles. The focus is on the linear quadratic stochastic optimal control problem for which a complete and novel theory is presented. Apart from the new algorithm, sample complexity bounds are obtained, and it is shown that the mean square error scales as $1/N$ where $N$ is the number of particles. The theoretical results and algorithms are illustrated with numerical experiments and comparisons with other recent approaches, where the faster convergence of the proposed algorithm is numerically demonstrated.

Cite this Paper

BibTeX
@InProceedings{pmlr-v283-joshi25a, title = {Interacting Particle Systems for Fast Linear Quadratic RL}, author = {Joshi, Anant A. and Chang, Heng-Sheng and Taghvaei, Amirhossein and Mehta, Prashant G. and Meyn, Sean P.}, booktitle = {Proceedings of the 7th Annual Learning for Dynamics \& Control Conference}, pages = {99--111}, year = {2025}, editor = {Ozay, Necmiye and Balzano, Laura and Panagou, Dimitra and Abate, Alessandro}, volume = {283}, series = {Proceedings of Machine Learning Research}, month = {04--06 Jun}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v283/main/assets/joshi25a/joshi25a.pdf}, url = {https://proceedings.mlr.press/v283/joshi25a.html}, abstract = {This paper is concerned with the design of algorithms based on systems of interacting particles to represent, approximate, and learn the optimal control law for reinforcement learning (RL). The primary contribution is that convergence rates are greatly accelerated by the interactions between particles. The focus is on the linear quadratic stochastic optimal control problem for which a complete and novel theory is presented. Apart from the new algorithm, sample complexity bounds are obtained, and it is shown that the mean square error scales as $1/N$ where $N$ is the number of particles. The theoretical results and algorithms are illustrated with numerical experiments and comparisons with other recent approaches, where the faster convergence of the proposed algorithm is numerically demonstrated.} }
Endnote
%0 Conference Paper %T Interacting Particle Systems for Fast Linear Quadratic RL %A Anant A. Joshi %A Heng-Sheng Chang %A Amirhossein Taghvaei %A Prashant G. Mehta %A Sean P. Meyn %B Proceedings of the 7th Annual Learning for Dynamics \& Control Conference %C Proceedings of Machine Learning Research %D 2025 %E Necmiye Ozay %E Laura Balzano %E Dimitra Panagou %E Alessandro Abate %F pmlr-v283-joshi25a %I PMLR %P 99--111 %U https://proceedings.mlr.press/v283/joshi25a.html %V 283 %X This paper is concerned with the design of algorithms based on systems of interacting particles to represent, approximate, and learn the optimal control law for reinforcement learning (RL). The primary contribution is that convergence rates are greatly accelerated by the interactions between particles. The focus is on the linear quadratic stochastic optimal control problem for which a complete and novel theory is presented. Apart from the new algorithm, sample complexity bounds are obtained, and it is shown that the mean square error scales as $1/N$ where $N$ is the number of particles. The theoretical results and algorithms are illustrated with numerical experiments and comparisons with other recent approaches, where the faster convergence of the proposed algorithm is numerically demonstrated.
APA
Joshi, A.A., Chang, H., Taghvaei, A., Mehta, P.G. & Meyn, S.P.. (2025). Interacting Particle Systems for Fast Linear Quadratic RL. Proceedings of the 7th Annual Learning for Dynamics \& Control Conference, in Proceedings of Machine Learning Research 283:99-111 Available from https://proceedings.mlr.press/v283/joshi25a.html.

Related Material