Hard Tasks First: Multi-Task Reinforcement Learning Through Task Scheduling

Myungsik Cho, Jongeui Park, Suyoung Lee, Youngchul Sung
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:8556-8577, 2024.

Abstract

Multi-task reinforcement learning (RL) faces the significant challenge of varying task difficulties, often leading to negative transfer when simpler tasks overshadow the learning of more complex ones. To overcome this challenge, we propose a novel algorithm, Scheduled Multi-Task Training (SMT), that strategically prioritizes more challenging tasks, thereby enhancing overall learning efficiency. SMT introduces a dynamic task prioritization strategy, underpinned by an effective metric for assessing task difficulty. This metric ensures an efficient and targeted allocation of training resources, significantly improving learning outcomes. Additionally, SMT incorporates a reset mechanism that periodically reinitializes key network parameters to mitigate the simplicity bias, further enhancing the adaptability and robustness of the learning process across diverse tasks. The efficacy of SMT’s scheduling method is validated by significantly improving performance on challenging Meta-World benchmarks.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-cho24d, title = {Hard Tasks First: Multi-Task Reinforcement Learning Through Task Scheduling}, author = {Cho, Myungsik and Park, Jongeui and Lee, Suyoung and Sung, Youngchul}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {8556--8577}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/cho24d/cho24d.pdf}, url = {https://proceedings.mlr.press/v235/cho24d.html}, abstract = {Multi-task reinforcement learning (RL) faces the significant challenge of varying task difficulties, often leading to negative transfer when simpler tasks overshadow the learning of more complex ones. To overcome this challenge, we propose a novel algorithm, Scheduled Multi-Task Training (SMT), that strategically prioritizes more challenging tasks, thereby enhancing overall learning efficiency. SMT introduces a dynamic task prioritization strategy, underpinned by an effective metric for assessing task difficulty. This metric ensures an efficient and targeted allocation of training resources, significantly improving learning outcomes. Additionally, SMT incorporates a reset mechanism that periodically reinitializes key network parameters to mitigate the simplicity bias, further enhancing the adaptability and robustness of the learning process across diverse tasks. The efficacy of SMT’s scheduling method is validated by significantly improving performance on challenging Meta-World benchmarks.} }
Endnote
%0 Conference Paper %T Hard Tasks First: Multi-Task Reinforcement Learning Through Task Scheduling %A Myungsik Cho %A Jongeui Park %A Suyoung Lee %A Youngchul Sung %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-cho24d %I PMLR %P 8556--8577 %U https://proceedings.mlr.press/v235/cho24d.html %V 235 %X Multi-task reinforcement learning (RL) faces the significant challenge of varying task difficulties, often leading to negative transfer when simpler tasks overshadow the learning of more complex ones. To overcome this challenge, we propose a novel algorithm, Scheduled Multi-Task Training (SMT), that strategically prioritizes more challenging tasks, thereby enhancing overall learning efficiency. SMT introduces a dynamic task prioritization strategy, underpinned by an effective metric for assessing task difficulty. This metric ensures an efficient and targeted allocation of training resources, significantly improving learning outcomes. Additionally, SMT incorporates a reset mechanism that periodically reinitializes key network parameters to mitigate the simplicity bias, further enhancing the adaptability and robustness of the learning process across diverse tasks. The efficacy of SMT’s scheduling method is validated by significantly improving performance on challenging Meta-World benchmarks.
APA
Cho, M., Park, J., Lee, S. & Sung, Y.. (2024). Hard Tasks First: Multi-Task Reinforcement Learning Through Task Scheduling. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:8556-8577 Available from https://proceedings.mlr.press/v235/cho24d.html.

Related Material