Learning-based Optimisation of Particle Accelerators Under Partial Observability Without Real-World Training

Jan Kaiser, Oliver Stein, Annika Eichler
Proceedings of the 39th International Conference on Machine Learning, PMLR 162:10575-10585, 2022.

Abstract

In recent work, it has been shown that reinforcement learning (RL) is capable of solving a variety of problems at sometimes super-human performance levels. But despite continued advances in the field, applying RL to complex real-world control and optimisation problems has proven difficult. In this contribution, we demonstrate how to successfully apply RL to the optimisation of a highly complex real-world machine {–} specifically a linear particle accelerator {–} in an only partially observable setting and without requiring training on the real machine. Our method outperforms conventional optimisation algorithms in both the achieved result and time taken as well as already achieving close to human-level performance. We expect that such automation of machine optimisation will push the limits of operability, increase machine availability and lead to a paradigm shift in how such machines are operated, ultimately facilitating advances in a variety of fields, such as science and medicine among many others.

Cite this Paper

BibTeX
@InProceedings{pmlr-v162-kaiser22a, title = {Learning-based Optimisation of Particle Accelerators Under Partial Observability Without Real-World Training}, author = {Kaiser, Jan and Stein, Oliver and Eichler, Annika}, booktitle = {Proceedings of the 39th International Conference on Machine Learning}, pages = {10575--10585}, year = {2022}, editor = {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan}, volume = {162}, series = {Proceedings of Machine Learning Research}, month = {17--23 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v162/kaiser22a/kaiser22a.pdf}, url = {https://proceedings.mlr.press/v162/kaiser22a.html}, abstract = {In recent work, it has been shown that reinforcement learning (RL) is capable of solving a variety of problems at sometimes super-human performance levels. But despite continued advances in the field, applying RL to complex real-world control and optimisation problems has proven difficult. In this contribution, we demonstrate how to successfully apply RL to the optimisation of a highly complex real-world machine {–} specifically a linear particle accelerator {–} in an only partially observable setting and without requiring training on the real machine. Our method outperforms conventional optimisation algorithms in both the achieved result and time taken as well as already achieving close to human-level performance. We expect that such automation of machine optimisation will push the limits of operability, increase machine availability and lead to a paradigm shift in how such machines are operated, ultimately facilitating advances in a variety of fields, such as science and medicine among many others.} }
Endnote
%0 Conference Paper %T Learning-based Optimisation of Particle Accelerators Under Partial Observability Without Real-World Training %A Jan Kaiser %A Oliver Stein %A Annika Eichler %B Proceedings of the 39th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2022 %E Kamalika Chaudhuri %E Stefanie Jegelka %E Le Song %E Csaba Szepesvari %E Gang Niu %E Sivan Sabato %F pmlr-v162-kaiser22a %I PMLR %P 10575--10585 %U https://proceedings.mlr.press/v162/kaiser22a.html %V 162 %X In recent work, it has been shown that reinforcement learning (RL) is capable of solving a variety of problems at sometimes super-human performance levels. But despite continued advances in the field, applying RL to complex real-world control and optimisation problems has proven difficult. In this contribution, we demonstrate how to successfully apply RL to the optimisation of a highly complex real-world machine {–} specifically a linear particle accelerator {–} in an only partially observable setting and without requiring training on the real machine. Our method outperforms conventional optimisation algorithms in both the achieved result and time taken as well as already achieving close to human-level performance. We expect that such automation of machine optimisation will push the limits of operability, increase machine availability and lead to a paradigm shift in how such machines are operated, ultimately facilitating advances in a variety of fields, such as science and medicine among many others.
APA
Kaiser, J., Stein, O. & Eichler, A.. (2022). Learning-based Optimisation of Particle Accelerators Under Partial Observability Without Real-World Training. Proceedings of the 39th International Conference on Machine Learning, in Proceedings of Machine Learning Research 162:10575-10585 Available from https://proceedings.mlr.press/v162/kaiser22a.html.

Related Material