Offline Reinforcement Learning for Rotation Profile Control in Tokamaks

Rohit Sonker, Hiro Josep Farre Kaga, Jiayu Chen, Andrew Rothstein, Ian Char, Ricardo Shousha, Egemen Kolemen, Jeff Schneider
Proceedings of The 8th Annual Learning for Dynamics and Control Conference, PMLR 331:904-924, 2026.

Abstract

Tokamaks remain leading candidates for achieving practical fusion energy, yet many important control problems inside these devices are still difficult or unsolved. One such challenge is controlling the plasma’s rotation profile, which strongly influences stability, confinement, and transport. While the average rotation can be controlled, controlling the full profile is challenging due to high dimensionality, response to multiple actuators and dependence on plasma condition. Learning based control, such as reinforcement learning (RL), provide a potential solution to this challenging problem with ability to model complex interactions leading to effective multi-input multi-output control. However, learning such policies is challenging due to the lack of accurate simulators which can model the rotation profile dynamics. In this work, we investigate the use of offline RL and offline model based RL algorithms for rotation profile control, training them solely on historical data from the DIII-D tokamak. Our final method uses probabilistic models of plasma dynamics to generate rollouts for RL training. We deploy this policy on the DIII-D Tokamak and observe promising real world results. We conclude by highlighting key challenges and insights from training and deploying an RL policy on a complex physical device while using only limited past data

Cite this Paper

BibTeX
@InProceedings{pmlr-v331-sonker26a, title = {Offline Reinforcement Learning for Rotation Profile Control in Tokamaks}, author = {Sonker, Rohit and Kaga, Hiro Josep Farre and Chen, Jiayu and Rothstein, Andrew and Char, Ian and Shousha, Ricardo and Kolemen, Egemen and Schneider, Jeff}, booktitle = {Proceedings of The 8th Annual Learning for Dynamics and Control Conference}, pages = {904--924}, year = {2026}, editor = {Sukhatme, Gaurav and Lindemann, Lars and Tu, Stephen and Wierman, Adam and Atanasov, Nikolay}, volume = {331}, series = {Proceedings of Machine Learning Research}, month = {17--19 Jun}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v331/main/assets/sonker26a/sonker26a.pdf}, url = {https://proceedings.mlr.press/v331/sonker26a.html}, abstract = {Tokamaks remain leading candidates for achieving practical fusion energy, yet many important control problems inside these devices are still difficult or unsolved. One such challenge is controlling the plasma’s rotation profile, which strongly influences stability, confinement, and transport. While the average rotation can be controlled, controlling the full profile is challenging due to high dimensionality, response to multiple actuators and dependence on plasma condition. Learning based control, such as reinforcement learning (RL), provide a potential solution to this challenging problem with ability to model complex interactions leading to effective multi-input multi-output control. However, learning such policies is challenging due to the lack of accurate simulators which can model the rotation profile dynamics. In this work, we investigate the use of offline RL and offline model based RL algorithms for rotation profile control, training them solely on historical data from the DIII-D tokamak. Our final method uses probabilistic models of plasma dynamics to generate rollouts for RL training. We deploy this policy on the DIII-D Tokamak and observe promising real world results. We conclude by highlighting key challenges and insights from training and deploying an RL policy on a complex physical device while using only limited past data} }
Endnote
%0 Conference Paper %T Offline Reinforcement Learning for Rotation Profile Control in Tokamaks %A Rohit Sonker %A Hiro Josep Farre Kaga %A Jiayu Chen %A Andrew Rothstein %A Ian Char %A Ricardo Shousha %A Egemen Kolemen %A Jeff Schneider %B Proceedings of The 8th Annual Learning for Dynamics and Control Conference %C Proceedings of Machine Learning Research %D 2026 %E Gaurav Sukhatme %E Lars Lindemann %E Stephen Tu %E Adam Wierman %E Nikolay Atanasov %F pmlr-v331-sonker26a %I PMLR %P 904--924 %U https://proceedings.mlr.press/v331/sonker26a.html %V 331 %X Tokamaks remain leading candidates for achieving practical fusion energy, yet many important control problems inside these devices are still difficult or unsolved. One such challenge is controlling the plasma’s rotation profile, which strongly influences stability, confinement, and transport. While the average rotation can be controlled, controlling the full profile is challenging due to high dimensionality, response to multiple actuators and dependence on plasma condition. Learning based control, such as reinforcement learning (RL), provide a potential solution to this challenging problem with ability to model complex interactions leading to effective multi-input multi-output control. However, learning such policies is challenging due to the lack of accurate simulators which can model the rotation profile dynamics. In this work, we investigate the use of offline RL and offline model based RL algorithms for rotation profile control, training them solely on historical data from the DIII-D tokamak. Our final method uses probabilistic models of plasma dynamics to generate rollouts for RL training. We deploy this policy on the DIII-D Tokamak and observe promising real world results. We conclude by highlighting key challenges and insights from training and deploying an RL policy on a complex physical device while using only limited past data
APA
Sonker, R., Kaga, H.J.F., Chen, J., Rothstein, A., Char, I., Shousha, R., Kolemen, E. & Schneider, J.. (2026). Offline Reinforcement Learning for Rotation Profile Control in Tokamaks. Proceedings of The 8th Annual Learning for Dynamics and Control Conference, in Proceedings of Machine Learning Research 331:904-924 Available from https://proceedings.mlr.press/v331/sonker26a.html.

Related Material