Reinforcing Real-world Service Agents: Balancing Utility and Cost in Task-oriented Dialogue

Ning Gao, Wei Zhang, Yuqin Dai, Ling Shi, Ziyin Wang, Yujie Wang, Wei He, Jinpeng Wang, Chaozheng Wang
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:33841-33878, 2026.

Abstract

The rapid evolution of Large Language Models (LLMs) has accelerated the transition from conversational chatbots to general agents. However, effectively balancing empathetic communication with budget-aware decision-making remains an open challenge. Since existing methods fail to capture these complex strategic trade-offs, we propose InteractCS-RL, a framework that reframes task-oriented dialogue as a multi-granularity reinforcement learning process. Specifically, we first establish a User-centric Interaction Framework to provide a high-fidelity training gym, enabling agents to dynamically explore diverse strategies with persona-driven users. Then, we introduce Cost-aware Multi-turn Policy Optimization (CMPO) with a hybrid advantage estimation strategy. By integrating generative process credits and employing a PID-Lagrangian cost controller, CMPO effectively guides the policy to explore Pareto boundary between user reward and global cost constraints. Extensive experiments on customized real business scenarios demonstrate that InteractCS-RL significantly outperform other baselines across three evaluation dimensions. Further evaluation on tool-agent-user interaction benchmarks verify InteractCS-RL robustness across diverse domains. The source code and data are available at https://github.com/NingGao-Ai/InteractCS-RL.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-gao26aj, title = {Reinforcing Real-world Service Agents: Balancing Utility and Cost in Task-oriented Dialogue}, author = {Gao, Ning and Zhang, Wei and Dai, Yuqin and Shi, Ling and Wang, Ziyin and Wang, Yujie and He, Wei and Wang, Jinpeng and Wang, Chaozheng}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {33841--33878}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/gao26aj/gao26aj.pdf}, url = {https://proceedings.mlr.press/v306/gao26aj.html}, abstract = {The rapid evolution of Large Language Models (LLMs) has accelerated the transition from conversational chatbots to general agents. However, effectively balancing empathetic communication with budget-aware decision-making remains an open challenge. Since existing methods fail to capture these complex strategic trade-offs, we propose InteractCS-RL, a framework that reframes task-oriented dialogue as a multi-granularity reinforcement learning process. Specifically, we first establish a User-centric Interaction Framework to provide a high-fidelity training gym, enabling agents to dynamically explore diverse strategies with persona-driven users. Then, we introduce Cost-aware Multi-turn Policy Optimization (CMPO) with a hybrid advantage estimation strategy. By integrating generative process credits and employing a PID-Lagrangian cost controller, CMPO effectively guides the policy to explore Pareto boundary between user reward and global cost constraints. Extensive experiments on customized real business scenarios demonstrate that InteractCS-RL significantly outperform other baselines across three evaluation dimensions. Further evaluation on tool-agent-user interaction benchmarks verify InteractCS-RL robustness across diverse domains. The source code and data are available at https://github.com/NingGao-Ai/InteractCS-RL.} }
Endnote
%0 Conference Paper %T Reinforcing Real-world Service Agents: Balancing Utility and Cost in Task-oriented Dialogue %A Ning Gao %A Wei Zhang %A Yuqin Dai %A Ling Shi %A Ziyin Wang %A Yujie Wang %A Wei He %A Jinpeng Wang %A Chaozheng Wang %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-gao26aj %I PMLR %P 33841--33878 %U https://proceedings.mlr.press/v306/gao26aj.html %V 306 %X The rapid evolution of Large Language Models (LLMs) has accelerated the transition from conversational chatbots to general agents. However, effectively balancing empathetic communication with budget-aware decision-making remains an open challenge. Since existing methods fail to capture these complex strategic trade-offs, we propose InteractCS-RL, a framework that reframes task-oriented dialogue as a multi-granularity reinforcement learning process. Specifically, we first establish a User-centric Interaction Framework to provide a high-fidelity training gym, enabling agents to dynamically explore diverse strategies with persona-driven users. Then, we introduce Cost-aware Multi-turn Policy Optimization (CMPO) with a hybrid advantage estimation strategy. By integrating generative process credits and employing a PID-Lagrangian cost controller, CMPO effectively guides the policy to explore Pareto boundary between user reward and global cost constraints. Extensive experiments on customized real business scenarios demonstrate that InteractCS-RL significantly outperform other baselines across three evaluation dimensions. Further evaluation on tool-agent-user interaction benchmarks verify InteractCS-RL robustness across diverse domains. The source code and data are available at https://github.com/NingGao-Ai/InteractCS-RL.
APA
Gao, N., Zhang, W., Dai, Y., Shi, L., Wang, Z., Wang, Y., He, W., Wang, J. & Wang, C.. (2026). Reinforcing Real-world Service Agents: Balancing Utility and Cost in Task-oriented Dialogue. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:33841-33878 Available from https://proceedings.mlr.press/v306/gao26aj.html.

Related Material