Black-Box Combinatorial Optimization with Order-Invariant Reinforcement Learning

Olivier Goudet, Quentin Suire, Adrien Goëffon, Frédéric Saubion, Sylvain Lamprier
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:36455-36503, 2026.

Abstract

We introduce an order-invariant reinforcement learning framework for black-box combinatorial optimization. Classical estimation-of-distribution algorithms (EDAs) often rely on learning explicit variable dependency graphs, which can be costly and may fail to capture complex interactions efficiently. In contrast, we parameterize a multivariate autoregressive generative model trained without a fixed variable ordering. By sampling random generation orders during training, a form of information-preserving dropout, the model is encouraged to be invariant to variable order, promoting search-space diversity, and shaping the model to focus on the most relevant variable dependencies, improving sample efficiency. We adapt Group Relative Policy Optimization (GRPO) to this setting, providing stable policy-gradient updates from scale-invariant advantages. Across a wide range of benchmark problem instances of varying sizes, our method frequently achieves the best performance and consistently avoids catastrophic failures.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-goudet26a, title = {Black-Box Combinatorial Optimization with Order-Invariant Reinforcement Learning}, author = {Goudet, Olivier and Suire, Quentin and Go\"{e}ffon, Adrien and Saubion, Fr\'{e}d\'{e}ric and Lamprier, Sylvain}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {36455--36503}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/goudet26a/goudet26a.pdf}, url = {https://proceedings.mlr.press/v306/goudet26a.html}, abstract = {We introduce an order-invariant reinforcement learning framework for black-box combinatorial optimization. Classical estimation-of-distribution algorithms (EDAs) often rely on learning explicit variable dependency graphs, which can be costly and may fail to capture complex interactions efficiently. In contrast, we parameterize a multivariate autoregressive generative model trained without a fixed variable ordering. By sampling random generation orders during training, a form of information-preserving dropout, the model is encouraged to be invariant to variable order, promoting search-space diversity, and shaping the model to focus on the most relevant variable dependencies, improving sample efficiency. We adapt Group Relative Policy Optimization (GRPO) to this setting, providing stable policy-gradient updates from scale-invariant advantages. Across a wide range of benchmark problem instances of varying sizes, our method frequently achieves the best performance and consistently avoids catastrophic failures.} }
Endnote
%0 Conference Paper %T Black-Box Combinatorial Optimization with Order-Invariant Reinforcement Learning %A Olivier Goudet %A Quentin Suire %A Adrien Goëffon %A Frédéric Saubion %A Sylvain Lamprier %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-goudet26a %I PMLR %P 36455--36503 %U https://proceedings.mlr.press/v306/goudet26a.html %V 306 %X We introduce an order-invariant reinforcement learning framework for black-box combinatorial optimization. Classical estimation-of-distribution algorithms (EDAs) often rely on learning explicit variable dependency graphs, which can be costly and may fail to capture complex interactions efficiently. In contrast, we parameterize a multivariate autoregressive generative model trained without a fixed variable ordering. By sampling random generation orders during training, a form of information-preserving dropout, the model is encouraged to be invariant to variable order, promoting search-space diversity, and shaping the model to focus on the most relevant variable dependencies, improving sample efficiency. We adapt Group Relative Policy Optimization (GRPO) to this setting, providing stable policy-gradient updates from scale-invariant advantages. Across a wide range of benchmark problem instances of varying sizes, our method frequently achieves the best performance and consistently avoids catastrophic failures.
APA
Goudet, O., Suire, Q., Goëffon, A., Saubion, F. & Lamprier, S.. (2026). Black-Box Combinatorial Optimization with Order-Invariant Reinforcement Learning. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:36455-36503 Available from https://proceedings.mlr.press/v306/goudet26a.html.

Related Material