HybridFlow: Resource-Adaptive Subtask Routing for Efficient Edge-Cloud LLM Inference

Jiangwen Dong, Jiayu Li, Tianhang Zheng, Wanyu Lin
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:25922-25943, 2026.

Abstract

Edge-cloud collaborative inference is crucial for LLM-powered edge devices, as on-device models often lack the required reasoning capability, while cloud-only inference can be costly and slow under strict latency and token/API budgets. However, existing edge-cloud collaboration methods typically route input tasks based on their estimated difficulty. These static, coarse heuristics overlook subtask dependencies, missing opportunities for parallel execution and budget-adaptive routing. To this end, we propose HybridFlow, a resource-adaptive edge-cloud inference framework that enables parallel execution of interdependent subtasks. Specifically, we build a dependency-aware DAG for each input task, facilitating concurrent execution of subtasks once their dependencies are resolved, thereby reducing end-to-end latency. Additionally, we propose a dynamic benefit–cost utility model, optimizing the trade-off between accuracy, token/API cost, and latency in real-time. This dynamic routing minimizes unnecessary cloud usage while preserving reasoning quality. Across GPQA, MMLU-Pro, AIME24, and LiveBench-Reasoning, HybridFlow improves the cost-accuracy trade-off, reducing latency and cloud API usage while maintaining competitive accuracy.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-dong26g, title = {{H}ybrid{F}low: Resource-Adaptive Subtask Routing for Efficient Edge-Cloud {LLM} Inference}, author = {Dong, Jiangwen and Li, Jiayu and Zheng, Tianhang and Lin, Wanyu}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {25922--25943}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/dong26g/dong26g.pdf}, url = {https://proceedings.mlr.press/v306/dong26g.html}, abstract = {Edge-cloud collaborative inference is crucial for LLM-powered edge devices, as on-device models often lack the required reasoning capability, while cloud-only inference can be costly and slow under strict latency and token/API budgets. However, existing edge-cloud collaboration methods typically route input tasks based on their estimated difficulty. These static, coarse heuristics overlook subtask dependencies, missing opportunities for parallel execution and budget-adaptive routing. To this end, we propose HybridFlow, a resource-adaptive edge-cloud inference framework that enables parallel execution of interdependent subtasks. Specifically, we build a dependency-aware DAG for each input task, facilitating concurrent execution of subtasks once their dependencies are resolved, thereby reducing end-to-end latency. Additionally, we propose a dynamic benefit–cost utility model, optimizing the trade-off between accuracy, token/API cost, and latency in real-time. This dynamic routing minimizes unnecessary cloud usage while preserving reasoning quality. Across GPQA, MMLU-Pro, AIME24, and LiveBench-Reasoning, HybridFlow improves the cost-accuracy trade-off, reducing latency and cloud API usage while maintaining competitive accuracy.} }
Endnote
%0 Conference Paper %T HybridFlow: Resource-Adaptive Subtask Routing for Efficient Edge-Cloud LLM Inference %A Jiangwen Dong %A Jiayu Li %A Tianhang Zheng %A Wanyu Lin %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-dong26g %I PMLR %P 25922--25943 %U https://proceedings.mlr.press/v306/dong26g.html %V 306 %X Edge-cloud collaborative inference is crucial for LLM-powered edge devices, as on-device models often lack the required reasoning capability, while cloud-only inference can be costly and slow under strict latency and token/API budgets. However, existing edge-cloud collaboration methods typically route input tasks based on their estimated difficulty. These static, coarse heuristics overlook subtask dependencies, missing opportunities for parallel execution and budget-adaptive routing. To this end, we propose HybridFlow, a resource-adaptive edge-cloud inference framework that enables parallel execution of interdependent subtasks. Specifically, we build a dependency-aware DAG for each input task, facilitating concurrent execution of subtasks once their dependencies are resolved, thereby reducing end-to-end latency. Additionally, we propose a dynamic benefit–cost utility model, optimizing the trade-off between accuracy, token/API cost, and latency in real-time. This dynamic routing minimizes unnecessary cloud usage while preserving reasoning quality. Across GPQA, MMLU-Pro, AIME24, and LiveBench-Reasoning, HybridFlow improves the cost-accuracy trade-off, reducing latency and cloud API usage while maintaining competitive accuracy.
APA
Dong, J., Li, J., Zheng, T. & Lin, W.. (2026). HybridFlow: Resource-Adaptive Subtask Routing for Efficient Edge-Cloud LLM Inference. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:25922-25943 Available from https://proceedings.mlr.press/v306/dong26g.html.

Related Material