ProOPF: Benchmarking and Improving LLMs for Professional-Grade Power Systems Optimization Modeling

Chao Shen, Zihan Guo, Xu Wan, Zhenghao Yang, Yifan Zhang, Wenqi Huang, Jie Song, Zongyan Zhang, Mingyang Sun
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:110912-110969, 2026.

Abstract

Growing renewable penetration introduces substantial uncertainty into power system operations, necessitating frequent adaptation of dispatch objectives and constraints and challenging expertise-intensive, near-real-time modeling workflows. Large Language Models (LLMs) provide a promising avenue for automating this process by translating natural-language (NL) operational requirements into executable optimization models via semantic reasoning and code synthesis. Yet existing LLM datasets and benchmarks for optimization modeling primarily target coarse-grained cross-domain generalization, offering limited, rigorous evaluation in power-system settings, particularly for Optimal Power Flow (OPF). We therefore introduce ProOPF-D and ProOPF-B, a dataset and benchmark for professional-grade OPF modeling: ProOPF-D contains 12K instances pairing NL requests with parameter adjustments and structural extensions to a canonical OPF, together with executable implementations; ProOPF-B provides 121 expert-annotated test cases with ground-truth code, enabling end-to-end evaluation under both concrete and abstract OPF modeling regimes. Our code, dataset, and benchmark are publicly available at https://github.com/shenchao188/ProOPF-Benchamrk-Dataset.

Cite this Paper


BibTeX
@InProceedings{pmlr-v306-shen26h, title = {{P}ro{OPF}: Benchmarking and Improving {LLM}s for Professional-Grade Power Systems Optimization Modeling}, author = {Shen, Chao and Guo, Zihan and Wan, Xu and Yang, Zhenghao and Zhang, Yifan and Huang, Wenqi and Song, Jie and Zhang, Zongyan and Sun, Mingyang}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {110912--110969}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/shen26h/shen26h.pdf}, url = {https://proceedings.mlr.press/v306/shen26h.html}, abstract = {Growing renewable penetration introduces substantial uncertainty into power system operations, necessitating frequent adaptation of dispatch objectives and constraints and challenging expertise-intensive, near-real-time modeling workflows. Large Language Models (LLMs) provide a promising avenue for automating this process by translating natural-language (NL) operational requirements into executable optimization models via semantic reasoning and code synthesis. Yet existing LLM datasets and benchmarks for optimization modeling primarily target coarse-grained cross-domain generalization, offering limited, rigorous evaluation in power-system settings, particularly for Optimal Power Flow (OPF). We therefore introduce ProOPF-D and ProOPF-B, a dataset and benchmark for professional-grade OPF modeling: ProOPF-D contains 12K instances pairing NL requests with parameter adjustments and structural extensions to a canonical OPF, together with executable implementations; ProOPF-B provides 121 expert-annotated test cases with ground-truth code, enabling end-to-end evaluation under both concrete and abstract OPF modeling regimes. Our code, dataset, and benchmark are publicly available at https://github.com/shenchao188/ProOPF-Benchamrk-Dataset.} }
Endnote
%0 Conference Paper %T ProOPF: Benchmarking and Improving LLMs for Professional-Grade Power Systems Optimization Modeling %A Chao Shen %A Zihan Guo %A Xu Wan %A Zhenghao Yang %A Yifan Zhang %A Wenqi Huang %A Jie Song %A Zongyan Zhang %A Mingyang Sun %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-shen26h %I PMLR %P 110912--110969 %U https://proceedings.mlr.press/v306/shen26h.html %V 306 %X Growing renewable penetration introduces substantial uncertainty into power system operations, necessitating frequent adaptation of dispatch objectives and constraints and challenging expertise-intensive, near-real-time modeling workflows. Large Language Models (LLMs) provide a promising avenue for automating this process by translating natural-language (NL) operational requirements into executable optimization models via semantic reasoning and code synthesis. Yet existing LLM datasets and benchmarks for optimization modeling primarily target coarse-grained cross-domain generalization, offering limited, rigorous evaluation in power-system settings, particularly for Optimal Power Flow (OPF). We therefore introduce ProOPF-D and ProOPF-B, a dataset and benchmark for professional-grade OPF modeling: ProOPF-D contains 12K instances pairing NL requests with parameter adjustments and structural extensions to a canonical OPF, together with executable implementations; ProOPF-B provides 121 expert-annotated test cases with ground-truth code, enabling end-to-end evaluation under both concrete and abstract OPF modeling regimes. Our code, dataset, and benchmark are publicly available at https://github.com/shenchao188/ProOPF-Benchamrk-Dataset.
APA
Shen, C., Guo, Z., Wan, X., Yang, Z., Zhang, Y., Huang, W., Song, J., Zhang, Z. & Sun, M.. (2026). ProOPF: Benchmarking and Improving LLMs for Professional-Grade Power Systems Optimization Modeling. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:110912-110969 Available from https://proceedings.mlr.press/v306/shen26h.html.

Related Material