Failure-Driven Workflow Refinement

Jusheng Zhang, Kaitong Cai, Jing Yang, Ziliang Chen, Yongsen Zheng, Kwok-Yan Lam, Liang Lin, Keze Wang
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:154214-154240, 2026.

Abstract

Workflow optimization for tool-using LLM agents is often cast as global search over candidate graphs, scored by a scalar metric. This collapses rich, multi-step failure traces into binary outcomes, obscuring recurring failure structure and making refinement inefficient. We reframe optimization as distributional refinement: each workflow induces a density over a Failure Signature Space $\mathcal{F}$, and the goal is to minimize its Expected Failure Mass. We propose CE-Graph, which maintains a counterexample pool, estimates dense failure modes, and applies operator-constrained graph edits via a Propose-and-Verify loop with a convergence-aware stopping rule. Across math, code, and QA benchmarks, CE-Graph improves robustness while reducing optimization cost compared to strong workflow-search baselines, suggesting reliability emerges from learning and reshaping failure landscapes rather than merely maximizing aggregate success rates.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-zhang26f, title = {Failure-Driven Workflow Refinement}, author = {Zhang, Jusheng and Cai, Kaitong and Yang, Jing and Chen, Ziliang and Zheng, Yongsen and Lam, Kwok-Yan and Lin, Liang and Wang, Keze}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {154214--154240}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/zhang26f/zhang26f.pdf}, url = {https://proceedings.mlr.press/v306/zhang26f.html}, abstract = {Workflow optimization for tool-using LLM agents is often cast as global search over candidate graphs, scored by a scalar metric. This collapses rich, multi-step failure traces into binary outcomes, obscuring recurring failure structure and making refinement inefficient. We reframe optimization as distributional refinement: each workflow induces a density over a Failure Signature Space $\mathcal{F}$, and the goal is to minimize its Expected Failure Mass. We propose CE-Graph, which maintains a counterexample pool, estimates dense failure modes, and applies operator-constrained graph edits via a Propose-and-Verify loop with a convergence-aware stopping rule. Across math, code, and QA benchmarks, CE-Graph improves robustness while reducing optimization cost compared to strong workflow-search baselines, suggesting reliability emerges from learning and reshaping failure landscapes rather than merely maximizing aggregate success rates.} }
Endnote
%0 Conference Paper %T Failure-Driven Workflow Refinement %A Jusheng Zhang %A Kaitong Cai %A Jing Yang %A Ziliang Chen %A Yongsen Zheng %A Kwok-Yan Lam %A Liang Lin %A Keze Wang %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-zhang26f %I PMLR %P 154214--154240 %U https://proceedings.mlr.press/v306/zhang26f.html %V 306 %X Workflow optimization for tool-using LLM agents is often cast as global search over candidate graphs, scored by a scalar metric. This collapses rich, multi-step failure traces into binary outcomes, obscuring recurring failure structure and making refinement inefficient. We reframe optimization as distributional refinement: each workflow induces a density over a Failure Signature Space $\mathcal{F}$, and the goal is to minimize its Expected Failure Mass. We propose CE-Graph, which maintains a counterexample pool, estimates dense failure modes, and applies operator-constrained graph edits via a Propose-and-Verify loop with a convergence-aware stopping rule. Across math, code, and QA benchmarks, CE-Graph improves robustness while reducing optimization cost compared to strong workflow-search baselines, suggesting reliability emerges from learning and reshaping failure landscapes rather than merely maximizing aggregate success rates.
APA
Zhang, J., Cai, K., Yang, J., Chen, Z., Zheng, Y., Lam, K., Lin, L. & Wang, K.. (2026). Failure-Driven Workflow Refinement. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:154214-154240 Available from https://proceedings.mlr.press/v306/zhang26f.html.

Related Material