$A_2$DEPT: Large Language Model–Driven Automated Algorithm Design via Evolutionary Program Trees

Bin Chen, Shouliang Zhu, Beidan Liu, Yong Zhao, Tianle Pu, Huichun Li, Zhengqiu Zhu
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:18654-18693, 2026.

Abstract

Designing heuristics for combinatorial optimization problems (COPs) is a fundamental yet challenging task that traditionally requires extensive domain expertise. Recently, Large Language Model (LLM)-based Automated Heuristic Design (AHD) has shown promise in autonomously generating heuristic components with minimal human intervention. However, most existing LLM-based AHD methods enforce fixed algorithmic templates to ensure executability, which confines the search to component-level tuning and limits system-level algorithmic expressiveness. To enable open-ended solver synthesis beyond rigid templates, we propose Automated Algorithm Design via Evolutionary Program Trees (A$_2$DEPT), which treats LLMs as system-level algorithm architects. A$_2$DEPT explores the vast program space via a tree-structured evolutionary search with hybrid selection and hierarchical operators, enabling iterative refinement of complete algorithms. To make open-ended generation practical, we enforce executability with a lightweight program-maintenance loop that performs feedback-driven repair. In experiments, A$_2$DEPT consistently outperforms state-of-the-art baselines across standard and highly constrained benchmarks, reducing the optimality gap by an average of 9.8%. Our work implies that system-level algorithm synthesis is a viable and scalable paradigm for LLM-driven optimization.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-chen26hb, title = {$A_2${DEPT}: Large Language Model–Driven Automated Algorithm Design via Evolutionary Program Trees}, author = {Chen, Bin and Zhu, Shouliang and Liu, Beidan and Zhao, Yong and Pu, Tianle and Li, Huichun and Zhu, Zhengqiu}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {18654--18693}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/chen26hb/chen26hb.pdf}, url = {https://proceedings.mlr.press/v306/chen26hb.html}, abstract = {Designing heuristics for combinatorial optimization problems (COPs) is a fundamental yet challenging task that traditionally requires extensive domain expertise. Recently, Large Language Model (LLM)-based Automated Heuristic Design (AHD) has shown promise in autonomously generating heuristic components with minimal human intervention. However, most existing LLM-based AHD methods enforce fixed algorithmic templates to ensure executability, which confines the search to component-level tuning and limits system-level algorithmic expressiveness. To enable open-ended solver synthesis beyond rigid templates, we propose Automated Algorithm Design via Evolutionary Program Trees (A$_2$DEPT), which treats LLMs as system-level algorithm architects. A$_2$DEPT explores the vast program space via a tree-structured evolutionary search with hybrid selection and hierarchical operators, enabling iterative refinement of complete algorithms. To make open-ended generation practical, we enforce executability with a lightweight program-maintenance loop that performs feedback-driven repair. In experiments, A$_2$DEPT consistently outperforms state-of-the-art baselines across standard and highly constrained benchmarks, reducing the optimality gap by an average of 9.8%. Our work implies that system-level algorithm synthesis is a viable and scalable paradigm for LLM-driven optimization.} }
Endnote
%0 Conference Paper %T $A_2$DEPT: Large Language Model–Driven Automated Algorithm Design via Evolutionary Program Trees %A Bin Chen %A Shouliang Zhu %A Beidan Liu %A Yong Zhao %A Tianle Pu %A Huichun Li %A Zhengqiu Zhu %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-chen26hb %I PMLR %P 18654--18693 %U https://proceedings.mlr.press/v306/chen26hb.html %V 306 %X Designing heuristics for combinatorial optimization problems (COPs) is a fundamental yet challenging task that traditionally requires extensive domain expertise. Recently, Large Language Model (LLM)-based Automated Heuristic Design (AHD) has shown promise in autonomously generating heuristic components with minimal human intervention. However, most existing LLM-based AHD methods enforce fixed algorithmic templates to ensure executability, which confines the search to component-level tuning and limits system-level algorithmic expressiveness. To enable open-ended solver synthesis beyond rigid templates, we propose Automated Algorithm Design via Evolutionary Program Trees (A$_2$DEPT), which treats LLMs as system-level algorithm architects. A$_2$DEPT explores the vast program space via a tree-structured evolutionary search with hybrid selection and hierarchical operators, enabling iterative refinement of complete algorithms. To make open-ended generation practical, we enforce executability with a lightweight program-maintenance loop that performs feedback-driven repair. In experiments, A$_2$DEPT consistently outperforms state-of-the-art baselines across standard and highly constrained benchmarks, reducing the optimality gap by an average of 9.8%. Our work implies that system-level algorithm synthesis is a viable and scalable paradigm for LLM-driven optimization.
APA
Chen, B., Zhu, S., Liu, B., Zhao, Y., Pu, T., Li, H. & Zhu, Z.. (2026). $A_2$DEPT: Large Language Model–Driven Automated Algorithm Design via Evolutionary Program Trees. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:18654-18693 Available from https://proceedings.mlr.press/v306/chen26hb.html.

Related Material