A Deep Reinforcement Learning Perspective on Internet Congestion Control

Nathan Jay, Noga Rotman, Brighten Godfrey, Michael Schapira, Aviv Tamar
Proceedings of the 36th International Conference on Machine Learning, PMLR 97:3050-3059, 2019.

Abstract

We present and investigate a novel and timely application domain for deep reinforcement learning (RL): Internet congestion control. Congestion control is the core networking task of modulating traffic sources’ data-transmission rates to efficiently utilize network capacity, and is the subject of extensive attention in light of the advent of Internet services such as live video, virtual reality, Internet-of-Things, and more. We show that casting congestion control as RL enables training deep network policies that capture intricate patterns in data traffic and network conditions, and leverage this to outperform the state-of-the-art. We also highlight significant challenges facing real-world adoption of RL-based congestion control, including fairness, safety, and generalization, which are not trivial to address within conventional RL formalism. To facilitate further research and reproducibility of our results, we present a test suite for RL-guided congestion control based on the OpenAI Gym interface.

Cite this Paper

BibTeX
@InProceedings{pmlr-v97-jay19a, title = {A Deep Reinforcement Learning Perspective on Internet Congestion Control}, author = {Jay, Nathan and Rotman, Noga and Godfrey, Brighten and Schapira, Michael and Tamar, Aviv}, booktitle = {Proceedings of the 36th International Conference on Machine Learning}, pages = {3050--3059}, year = {2019}, editor = {Chaudhuri, Kamalika and Salakhutdinov, Ruslan}, volume = {97}, series = {Proceedings of Machine Learning Research}, month = {09--15 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v97/jay19a/jay19a.pdf}, url = {https://proceedings.mlr.press/v97/jay19a.html}, abstract = {We present and investigate a novel and timely application domain for deep reinforcement learning (RL): Internet congestion control. Congestion control is the core networking task of modulating traffic sources’ data-transmission rates to efficiently utilize network capacity, and is the subject of extensive attention in light of the advent of Internet services such as live video, virtual reality, Internet-of-Things, and more. We show that casting congestion control as RL enables training deep network policies that capture intricate patterns in data traffic and network conditions, and leverage this to outperform the state-of-the-art. We also highlight significant challenges facing real-world adoption of RL-based congestion control, including fairness, safety, and generalization, which are not trivial to address within conventional RL formalism. To facilitate further research and reproducibility of our results, we present a test suite for RL-guided congestion control based on the OpenAI Gym interface.} }
Endnote
%0 Conference Paper %T A Deep Reinforcement Learning Perspective on Internet Congestion Control %A Nathan Jay %A Noga Rotman %A Brighten Godfrey %A Michael Schapira %A Aviv Tamar %B Proceedings of the 36th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2019 %E Kamalika Chaudhuri %E Ruslan Salakhutdinov %F pmlr-v97-jay19a %I PMLR %P 3050--3059 %U https://proceedings.mlr.press/v97/jay19a.html %V 97 %X We present and investigate a novel and timely application domain for deep reinforcement learning (RL): Internet congestion control. Congestion control is the core networking task of modulating traffic sources’ data-transmission rates to efficiently utilize network capacity, and is the subject of extensive attention in light of the advent of Internet services such as live video, virtual reality, Internet-of-Things, and more. We show that casting congestion control as RL enables training deep network policies that capture intricate patterns in data traffic and network conditions, and leverage this to outperform the state-of-the-art. We also highlight significant challenges facing real-world adoption of RL-based congestion control, including fairness, safety, and generalization, which are not trivial to address within conventional RL formalism. To facilitate further research and reproducibility of our results, we present a test suite for RL-guided congestion control based on the OpenAI Gym interface.
APA
Jay, N., Rotman, N., Godfrey, B., Schapira, M. & Tamar, A.. (2019). A Deep Reinforcement Learning Perspective on Internet Congestion Control. Proceedings of the 36th International Conference on Machine Learning, in Proceedings of Machine Learning Research 97:3050-3059 Available from https://proceedings.mlr.press/v97/jay19a.html.

Related Material