Inductive-bias-driven Reinforcement Learning For Efficient Schedules in Heterogeneous Clusters

Subho Banerjee, Saurabh Jha, Zbigniew Kalbarczyk, Ravishankar Iyer
Proceedings of the 37th International Conference on Machine Learning, PMLR 119:629-641, 2020.

Abstract

The problem of scheduling of workloads onto heterogeneous processors (e.g., CPUs, GPUs, FPGAs) is of fundamental importance in modern data centers. Current system schedulers rely on application/system-specific heuristics that have to be built on a case-by-case basis. Recent work has demonstrated ML techniques for automating the heuristic search by using black-box approaches which require significant training data and time, which make them challenging to use in practice. This paper presents Symphony, a scheduling framework that addresses the challenge in two ways: (i) a domain-driven Bayesian reinforcement learning (RL) model for scheduling, which inherently models the resource dependencies identified from the system architecture; and (ii) a sampling-based technique to compute the gradients of a Bayesian model without performing full probabilistic inference. Together, these techniques reduce both the amount of training data and the time required to produce scheduling policies that significantly outperform black-box approaches by up to 2.2{\texttimes}.

Cite this Paper

BibTeX
@InProceedings{pmlr-v119-banerjee20a, title = {Inductive-bias-driven Reinforcement Learning For Efficient Schedules in Heterogeneous Clusters}, author = {Banerjee, Subho and Jha, Saurabh and Kalbarczyk, Zbigniew and Iyer, Ravishankar}, booktitle = {Proceedings of the 37th International Conference on Machine Learning}, pages = {629--641}, year = {2020}, editor = {III, Hal Daumé and Singh, Aarti}, volume = {119}, series = {Proceedings of Machine Learning Research}, month = {13--18 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v119/banerjee20a/banerjee20a.pdf}, url = {https://proceedings.mlr.press/v119/banerjee20a.html}, abstract = {The problem of scheduling of workloads onto heterogeneous processors (e.g., CPUs, GPUs, FPGAs) is of fundamental importance in modern data centers. Current system schedulers rely on application/system-specific heuristics that have to be built on a case-by-case basis. Recent work has demonstrated ML techniques for automating the heuristic search by using black-box approaches which require significant training data and time, which make them challenging to use in practice. This paper presents Symphony, a scheduling framework that addresses the challenge in two ways: (i) a domain-driven Bayesian reinforcement learning (RL) model for scheduling, which inherently models the resource dependencies identified from the system architecture; and (ii) a sampling-based technique to compute the gradients of a Bayesian model without performing full probabilistic inference. Together, these techniques reduce both the amount of training data and the time required to produce scheduling policies that significantly outperform black-box approaches by up to 2.2{\texttimes}.} }
Endnote
%0 Conference Paper %T Inductive-bias-driven Reinforcement Learning For Efficient Schedules in Heterogeneous Clusters %A Subho Banerjee %A Saurabh Jha %A Zbigniew Kalbarczyk %A Ravishankar Iyer %B Proceedings of the 37th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2020 %E Hal Daumé III %E Aarti Singh %F pmlr-v119-banerjee20a %I PMLR %P 629--641 %U https://proceedings.mlr.press/v119/banerjee20a.html %V 119 %X The problem of scheduling of workloads onto heterogeneous processors (e.g., CPUs, GPUs, FPGAs) is of fundamental importance in modern data centers. Current system schedulers rely on application/system-specific heuristics that have to be built on a case-by-case basis. Recent work has demonstrated ML techniques for automating the heuristic search by using black-box approaches which require significant training data and time, which make them challenging to use in practice. This paper presents Symphony, a scheduling framework that addresses the challenge in two ways: (i) a domain-driven Bayesian reinforcement learning (RL) model for scheduling, which inherently models the resource dependencies identified from the system architecture; and (ii) a sampling-based technique to compute the gradients of a Bayesian model without performing full probabilistic inference. Together, these techniques reduce both the amount of training data and the time required to produce scheduling policies that significantly outperform black-box approaches by up to 2.2{\texttimes}.
APA
Banerjee, S., Jha, S., Kalbarczyk, Z. & Iyer, R.. (2020). Inductive-bias-driven Reinforcement Learning For Efficient Schedules in Heterogeneous Clusters. Proceedings of the 37th International Conference on Machine Learning, in Proceedings of Machine Learning Research 119:629-641 Available from https://proceedings.mlr.press/v119/banerjee20a.html.

Related Material