CoVO-MPC: Theoretical analysis of sampling-based MPC and optimal covariance design

Zeji Yi, Chaoyi Pan, Guanqi He, Guannan Qu, Guanya Shi
Proceedings of the 6th Annual Learning for Dynamics & Control Conference, PMLR 242:1122-1135, 2024.

Abstract

Sampling-based Model Predictive Control (MPC) has been a practical and effective approach in many domains, notably model-based reinforcement learning, thanks to its flexibility and parallelizability. Despite its appealing empirical performance, the theoretical understanding, particularly in terms of convergence analysis and hyperparameter tuning, remains absent. In this paper, we characterize the convergence property of a widely used sampling-based MPC method, Model Predictive Path Integral Control (MPPI). We show that MPPI enjoys at least linear convergence rates when the optimization is quadratic, which covers time-varying LQR systems. We then extend to more general nonlinear systems. Our theoretical analysis directly leads to a novel sampling-based MPC algorithm, CoVariance-Optimal MPC (CoVO-MPC) that optimally schedules the sampling covariance to optimize the convergence rate. Empirically, CoVO-MPC significantly outperforms standard MPPI by 43-54% in both simulations and real-world quadrotor agile control tasks. Videos and Appendices are available at https://tinyurl.com/covo-mpc-cmu.

Cite this Paper

BibTeX
@InProceedings{pmlr-v242-yi24b, title = {{CoVO}-{MPC}: {T}heoretical analysis of sampling-based {MPC} and optimal covariance design}, author = {Yi, Zeji and Pan, Chaoyi and He, Guanqi and Qu, Guannan and Shi, Guanya}, booktitle = {Proceedings of the 6th Annual Learning for Dynamics & Control Conference}, pages = {1122--1135}, year = {2024}, editor = {Abate, Alessandro and Cannon, Mark and Margellos, Kostas and Papachristodoulou, Antonis}, volume = {242}, series = {Proceedings of Machine Learning Research}, month = {15--17 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v242/yi24b/yi24b.pdf}, url = {https://proceedings.mlr.press/v242/yi24b.html}, abstract = {Sampling-based Model Predictive Control (MPC) has been a practical and effective approach in many domains, notably model-based reinforcement learning, thanks to its flexibility and parallelizability. Despite its appealing empirical performance, the theoretical understanding, particularly in terms of convergence analysis and hyperparameter tuning, remains absent. In this paper, we characterize the convergence property of a widely used sampling-based MPC method, Model Predictive Path Integral Control (MPPI). We show that MPPI enjoys at least linear convergence rates when the optimization is quadratic, which covers time-varying LQR systems. We then extend to more general nonlinear systems. Our theoretical analysis directly leads to a novel sampling-based MPC algorithm, CoVariance-Optimal MPC (CoVO-MPC) that optimally schedules the sampling covariance to optimize the convergence rate. Empirically, CoVO-MPC significantly outperforms standard MPPI by 43-54% in both simulations and real-world quadrotor agile control tasks. Videos and Appendices are available at https://tinyurl.com/covo-mpc-cmu.} }
Endnote
%0 Conference Paper %T CoVO-MPC: Theoretical analysis of sampling-based MPC and optimal covariance design %A Zeji Yi %A Chaoyi Pan %A Guanqi He %A Guannan Qu %A Guanya Shi %B Proceedings of the 6th Annual Learning for Dynamics & Control Conference %C Proceedings of Machine Learning Research %D 2024 %E Alessandro Abate %E Mark Cannon %E Kostas Margellos %E Antonis Papachristodoulou %F pmlr-v242-yi24b %I PMLR %P 1122--1135 %U https://proceedings.mlr.press/v242/yi24b.html %V 242 %X Sampling-based Model Predictive Control (MPC) has been a practical and effective approach in many domains, notably model-based reinforcement learning, thanks to its flexibility and parallelizability. Despite its appealing empirical performance, the theoretical understanding, particularly in terms of convergence analysis and hyperparameter tuning, remains absent. In this paper, we characterize the convergence property of a widely used sampling-based MPC method, Model Predictive Path Integral Control (MPPI). We show that MPPI enjoys at least linear convergence rates when the optimization is quadratic, which covers time-varying LQR systems. We then extend to more general nonlinear systems. Our theoretical analysis directly leads to a novel sampling-based MPC algorithm, CoVariance-Optimal MPC (CoVO-MPC) that optimally schedules the sampling covariance to optimize the convergence rate. Empirically, CoVO-MPC significantly outperforms standard MPPI by 43-54% in both simulations and real-world quadrotor agile control tasks. Videos and Appendices are available at https://tinyurl.com/covo-mpc-cmu.
APA
Yi, Z., Pan, C., He, G., Qu, G. & Shi, G.. (2024). CoVO-MPC: Theoretical analysis of sampling-based MPC and optimal covariance design. Proceedings of the 6th Annual Learning for Dynamics & Control Conference, in Proceedings of Machine Learning Research 242:1122-1135 Available from https://proceedings.mlr.press/v242/yi24b.html.

Related Material