Multi-Agent Reinforcement Learning Meets Leaf Sequencing in Radiotherapy

Riqiang Gao, Florin-Cristian Ghesu, Simon Arberet, Shahab Basiri, Esa Kuusela, Martin Kraus, Dorin Comaniciu, Ali Kamen
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:14723-14746, 2024.

Abstract

In contemporary radiotherapy planning (RTP), a key module leaf sequencing is predominantly addressed by optimization-based approaches. In this paper, we propose a novel deep reinforcement learning (DRL) model termed as Reinforced Leaf Sequencer (RLS) in a multi-agent framework for leaf sequencing. The RLS model offers improvements to time-consuming iterative optimization steps via large-scale training and can control movement patterns through the design of reward mechanisms. We have conducted experiments on four datasets with four metrics and compared our model with a leading optimization sequencer. Our findings reveal that the proposed RLS model can achieve reduced fluence reconstruction errors, and potential faster convergence when integrated in an optimization planner. Additionally, RLS has shown promising results in a full artificial intelligence RTP pipeline. We hope this pioneer multi-agent RL leaf sequencer can foster future research on machine learning for RTP.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-gao24g, title = {Multi-Agent Reinforcement Learning Meets Leaf Sequencing in Radiotherapy}, author = {Gao, Riqiang and Ghesu, Florin-Cristian and Arberet, Simon and Basiri, Shahab and Kuusela, Esa and Kraus, Martin and Comaniciu, Dorin and Kamen, Ali}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {14723--14746}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/gao24g/gao24g.pdf}, url = {https://proceedings.mlr.press/v235/gao24g.html}, abstract = {In contemporary radiotherapy planning (RTP), a key module leaf sequencing is predominantly addressed by optimization-based approaches. In this paper, we propose a novel deep reinforcement learning (DRL) model termed as Reinforced Leaf Sequencer (RLS) in a multi-agent framework for leaf sequencing. The RLS model offers improvements to time-consuming iterative optimization steps via large-scale training and can control movement patterns through the design of reward mechanisms. We have conducted experiments on four datasets with four metrics and compared our model with a leading optimization sequencer. Our findings reveal that the proposed RLS model can achieve reduced fluence reconstruction errors, and potential faster convergence when integrated in an optimization planner. Additionally, RLS has shown promising results in a full artificial intelligence RTP pipeline. We hope this pioneer multi-agent RL leaf sequencer can foster future research on machine learning for RTP.} }
Endnote
%0 Conference Paper %T Multi-Agent Reinforcement Learning Meets Leaf Sequencing in Radiotherapy %A Riqiang Gao %A Florin-Cristian Ghesu %A Simon Arberet %A Shahab Basiri %A Esa Kuusela %A Martin Kraus %A Dorin Comaniciu %A Ali Kamen %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-gao24g %I PMLR %P 14723--14746 %U https://proceedings.mlr.press/v235/gao24g.html %V 235 %X In contemporary radiotherapy planning (RTP), a key module leaf sequencing is predominantly addressed by optimization-based approaches. In this paper, we propose a novel deep reinforcement learning (DRL) model termed as Reinforced Leaf Sequencer (RLS) in a multi-agent framework for leaf sequencing. The RLS model offers improvements to time-consuming iterative optimization steps via large-scale training and can control movement patterns through the design of reward mechanisms. We have conducted experiments on four datasets with four metrics and compared our model with a leading optimization sequencer. Our findings reveal that the proposed RLS model can achieve reduced fluence reconstruction errors, and potential faster convergence when integrated in an optimization planner. Additionally, RLS has shown promising results in a full artificial intelligence RTP pipeline. We hope this pioneer multi-agent RL leaf sequencer can foster future research on machine learning for RTP.
APA
Gao, R., Ghesu, F., Arberet, S., Basiri, S., Kuusela, E., Kraus, M., Comaniciu, D. & Kamen, A.. (2024). Multi-Agent Reinforcement Learning Meets Leaf Sequencing in Radiotherapy. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:14723-14746 Available from https://proceedings.mlr.press/v235/gao24g.html.

Related Material