IEC: When Information-Driven Exploration Meets Spectral Consensus via Primal–Dual Reward Regularization in Decentralized MARL

Xuefeng Du, Jiajun Wu, Yuduo Zheng, Fengqi Li
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:26763-26796, 2026.

Abstract

Decentralized multi-agent reinforcement learning faces a persistent exploration–coordination tension: intrinsic rewards promote exploration under sparse feedback, yet effective cooperation requires agents’ behaviors to remain consistent over a limited communication graph. Existing methods often combine exploration bonuses and coordination regularizers with fixed-weight schedules, making them hard to tune and prone to either fragmented conventions or premature behavioral collapse. We propose the IEC (Isomorphic Exploration-Consensus) framework that couples exploration and coordination through a single constrained objective: maximize task return augmented with two complementary exploration signals, dynamics-based information gain and state-coverage novelty, while constraining graph-induced policy disagreement via a spectral smoothness penalty on neighboring agents, which can be interpreted as a Dirichlet-energy regularizer on the communication graph. IEC optimizes the resulting Lagrangian with a lightweight primal–dual update that adapts the consensus multiplier from observed constraint violations, yielding an automatic shift from diverse exploration to stable cooperative conventions. Across three distinct benchmarks, IEC achieves superior performance.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-du26p, title = {{IEC}: When Information-Driven Exploration Meets Spectral Consensus via Primal–Dual Reward Regularization in Decentralized {MARL}}, author = {Du, Xuefeng and Wu, Jiajun and Zheng, Yuduo and Li, Fengqi}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {26763--26796}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/du26p/du26p.pdf}, url = {https://proceedings.mlr.press/v306/du26p.html}, abstract = {Decentralized multi-agent reinforcement learning faces a persistent exploration–coordination tension: intrinsic rewards promote exploration under sparse feedback, yet effective cooperation requires agents’ behaviors to remain consistent over a limited communication graph. Existing methods often combine exploration bonuses and coordination regularizers with fixed-weight schedules, making them hard to tune and prone to either fragmented conventions or premature behavioral collapse. We propose the IEC (Isomorphic Exploration-Consensus) framework that couples exploration and coordination through a single constrained objective: maximize task return augmented with two complementary exploration signals, dynamics-based information gain and state-coverage novelty, while constraining graph-induced policy disagreement via a spectral smoothness penalty on neighboring agents, which can be interpreted as a Dirichlet-energy regularizer on the communication graph. IEC optimizes the resulting Lagrangian with a lightweight primal–dual update that adapts the consensus multiplier from observed constraint violations, yielding an automatic shift from diverse exploration to stable cooperative conventions. Across three distinct benchmarks, IEC achieves superior performance.} }
Endnote
%0 Conference Paper %T IEC: When Information-Driven Exploration Meets Spectral Consensus via Primal–Dual Reward Regularization in Decentralized MARL %A Xuefeng Du %A Jiajun Wu %A Yuduo Zheng %A Fengqi Li %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-du26p %I PMLR %P 26763--26796 %U https://proceedings.mlr.press/v306/du26p.html %V 306 %X Decentralized multi-agent reinforcement learning faces a persistent exploration–coordination tension: intrinsic rewards promote exploration under sparse feedback, yet effective cooperation requires agents’ behaviors to remain consistent over a limited communication graph. Existing methods often combine exploration bonuses and coordination regularizers with fixed-weight schedules, making them hard to tune and prone to either fragmented conventions or premature behavioral collapse. We propose the IEC (Isomorphic Exploration-Consensus) framework that couples exploration and coordination through a single constrained objective: maximize task return augmented with two complementary exploration signals, dynamics-based information gain and state-coverage novelty, while constraining graph-induced policy disagreement via a spectral smoothness penalty on neighboring agents, which can be interpreted as a Dirichlet-energy regularizer on the communication graph. IEC optimizes the resulting Lagrangian with a lightweight primal–dual update that adapts the consensus multiplier from observed constraint violations, yielding an automatic shift from diverse exploration to stable cooperative conventions. Across three distinct benchmarks, IEC achieves superior performance.
APA
Du, X., Wu, J., Zheng, Y. & Li, F.. (2026). IEC: When Information-Driven Exploration Meets Spectral Consensus via Primal–Dual Reward Regularization in Decentralized MARL. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:26763-26796 Available from https://proceedings.mlr.press/v306/du26p.html.

Related Material