PDE control gym: A benchmark for data-driven boundary control of partial differential equations

Luke Bhan, Yuexin Bian, Miroslav Krstic, Yuanyuan Shi
Proceedings of the 6th Annual Learning for Dynamics & Control Conference, PMLR 242:1083-1095, 2024.

Abstract

Over the last decade, data-driven methods have surged in popularity, emerging as valuable tools for control theory. As such, neural network approximations of control feedback laws, system dynamics, and even Lyapunov functions have attracted growing attention. With the ascent of learning based control, the need for accurate, fast, and easy-to-use benchmarks has increased. In this work, we present the first learning-based environment for boundary control of PDEs. In our benchmark, we introduce three foundational PDE problems — a 1D transport PDE, a 1D reaction-diffusion PDE, and a 2D Navier–Stokes PDE — whose solvers are bundled in an user-friendly reinforcement learning gym. With this gym, we then present the first set of model-free, reinforcement learning algorithms for solving this series of benchmark problems, achieving stability, although at a higher cost compared to model-based PDE backstepping. With the set of benchmark environments and detailed examples, this work significantly lowers the barrier to entry for learning-based PDE control — a topic largely unexplored by the data-driven control community. The entire benchmark is available on Github along with detailed documentation and the presented reinforcement learning models are open sourced.

Cite this Paper

BibTeX
@InProceedings{pmlr-v242-bhan24a, title = {{PDE} control gym: {A} benchmark for data-driven boundary control of partial differential equations}, author = {Bhan, Luke and Bian, Yuexin and Krstic, Miroslav and Shi, Yuanyuan}, booktitle = {Proceedings of the 6th Annual Learning for Dynamics & Control Conference}, pages = {1083--1095}, year = {2024}, editor = {Abate, Alessandro and Cannon, Mark and Margellos, Kostas and Papachristodoulou, Antonis}, volume = {242}, series = {Proceedings of Machine Learning Research}, month = {15--17 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v242/bhan24a/bhan24a.pdf}, url = {https://proceedings.mlr.press/v242/bhan24a.html}, abstract = {Over the last decade, data-driven methods have surged in popularity, emerging as valuable tools for control theory. As such, neural network approximations of control feedback laws, system dynamics, and even Lyapunov functions have attracted growing attention. With the ascent of learning based control, the need for accurate, fast, and easy-to-use benchmarks has increased. In this work, we present the first learning-based environment for boundary control of PDEs. In our benchmark, we introduce three foundational PDE problems — a 1D transport PDE, a 1D reaction-diffusion PDE, and a 2D Navier–Stokes PDE — whose solvers are bundled in an user-friendly reinforcement learning gym. With this gym, we then present the first set of model-free, reinforcement learning algorithms for solving this series of benchmark problems, achieving stability, although at a higher cost compared to model-based PDE backstepping. With the set of benchmark environments and detailed examples, this work significantly lowers the barrier to entry for learning-based PDE control — a topic largely unexplored by the data-driven control community. The entire benchmark is available on Github along with detailed documentation and the presented reinforcement learning models are open sourced.} }
Endnote
%0 Conference Paper %T PDE control gym: A benchmark for data-driven boundary control of partial differential equations %A Luke Bhan %A Yuexin Bian %A Miroslav Krstic %A Yuanyuan Shi %B Proceedings of the 6th Annual Learning for Dynamics & Control Conference %C Proceedings of Machine Learning Research %D 2024 %E Alessandro Abate %E Mark Cannon %E Kostas Margellos %E Antonis Papachristodoulou %F pmlr-v242-bhan24a %I PMLR %P 1083--1095 %U https://proceedings.mlr.press/v242/bhan24a.html %V 242 %X Over the last decade, data-driven methods have surged in popularity, emerging as valuable tools for control theory. As such, neural network approximations of control feedback laws, system dynamics, and even Lyapunov functions have attracted growing attention. With the ascent of learning based control, the need for accurate, fast, and easy-to-use benchmarks has increased. In this work, we present the first learning-based environment for boundary control of PDEs. In our benchmark, we introduce three foundational PDE problems — a 1D transport PDE, a 1D reaction-diffusion PDE, and a 2D Navier–Stokes PDE — whose solvers are bundled in an user-friendly reinforcement learning gym. With this gym, we then present the first set of model-free, reinforcement learning algorithms for solving this series of benchmark problems, achieving stability, although at a higher cost compared to model-based PDE backstepping. With the set of benchmark environments and detailed examples, this work significantly lowers the barrier to entry for learning-based PDE control — a topic largely unexplored by the data-driven control community. The entire benchmark is available on Github along with detailed documentation and the presented reinforcement learning models are open sourced.
APA
Bhan, L., Bian, Y., Krstic, M. & Shi, Y.. (2024). PDE control gym: A benchmark for data-driven boundary control of partial differential equations. Proceedings of the 6th Annual Learning for Dynamics & Control Conference, in Proceedings of Machine Learning Research 242:1083-1095 Available from https://proceedings.mlr.press/v242/bhan24a.html.

Related Material