Learning with Adaptive Resource Allocation

Jing Wang, Miao Yu, Peng Zhao, Zhi-Hua Zhou
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:52099-52116, 2024.

Abstract

The study of machine learning under limited resources has gathered increasing attention, considering improving the learning efficiency and effectiveness with budgeted resources. However, previous efforts mainly focus on single learning task, and a common resource-limited scenario is less explored: to handle multiple time-constrained learning tasks concurrently with budgeted computational resources. In this paper, we point out that this is a very challenging task because it demands the learner to be concerned about not only the progress of the learning tasks but also the coordinative allocation of computational resources. We present the Learning with Adaptive Resource Allocation (LARA) approach, which comprises an efficient online estimator for learning progress prediction, an adaptive search method for computational resource allocation, and a balancing strategy for alleviating prediction-allocation compounding errors. Empirical studies validate the effectiveness of our proposed approach.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-wang24cj, title = {Learning with Adaptive Resource Allocation}, author = {Wang, Jing and Yu, Miao and Zhao, Peng and Zhou, Zhi-Hua}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {52099--52116}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/wang24cj/wang24cj.pdf}, url = {https://proceedings.mlr.press/v235/wang24cj.html}, abstract = {The study of machine learning under limited resources has gathered increasing attention, considering improving the learning efficiency and effectiveness with budgeted resources. However, previous efforts mainly focus on single learning task, and a common resource-limited scenario is less explored: to handle multiple time-constrained learning tasks concurrently with budgeted computational resources. In this paper, we point out that this is a very challenging task because it demands the learner to be concerned about not only the progress of the learning tasks but also the coordinative allocation of computational resources. We present the Learning with Adaptive Resource Allocation (LARA) approach, which comprises an efficient online estimator for learning progress prediction, an adaptive search method for computational resource allocation, and a balancing strategy for alleviating prediction-allocation compounding errors. Empirical studies validate the effectiveness of our proposed approach.} }
Endnote
%0 Conference Paper %T Learning with Adaptive Resource Allocation %A Jing Wang %A Miao Yu %A Peng Zhao %A Zhi-Hua Zhou %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-wang24cj %I PMLR %P 52099--52116 %U https://proceedings.mlr.press/v235/wang24cj.html %V 235 %X The study of machine learning under limited resources has gathered increasing attention, considering improving the learning efficiency and effectiveness with budgeted resources. However, previous efforts mainly focus on single learning task, and a common resource-limited scenario is less explored: to handle multiple time-constrained learning tasks concurrently with budgeted computational resources. In this paper, we point out that this is a very challenging task because it demands the learner to be concerned about not only the progress of the learning tasks but also the coordinative allocation of computational resources. We present the Learning with Adaptive Resource Allocation (LARA) approach, which comprises an efficient online estimator for learning progress prediction, an adaptive search method for computational resource allocation, and a balancing strategy for alleviating prediction-allocation compounding errors. Empirical studies validate the effectiveness of our proposed approach.
APA
Wang, J., Yu, M., Zhao, P. & Zhou, Z.. (2024). Learning with Adaptive Resource Allocation. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:52099-52116 Available from https://proceedings.mlr.press/v235/wang24cj.html.

Related Material