HyPOLE: Hyperproperty-Guided Multi-Agent Reinforcement Learning under Partial Observation

Arshia Rafieioskouei, Tzu-Han Hsu, Matthew Lucas, Borzoo Bonakdarpour
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:102974-103001, 2026.

Abstract

Formal specification is a powerful tool to guide the learning process and provides significant advantages over reward shaping: (1) mathematical rigor; (2) expressiveness to specify objectives and constraints, and (3) the ability to define tactics to achieve objectives. However, these benefits remain largely unexplored in the context of Multi-Agent Reinforcement Learning (MARL). This paper introduces HyPOLE, a novel framework for MARL under partial observability, where learning is guided by the expressive power of the so-called hyperproperties and, in particular, the temporal logic HyperLTL. We integrate Centralized Training for Decentralized Execution (CTDE) techniques with HyPOLE to synthesize decentralized policies, and our evaluation on SMAC, MessySMAC, and WildFire benchmark demonstrates clear advantages over baselines.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-rafieioskouei26a, title = {{H}y{POLE}: Hyperproperty-Guided Multi-Agent Reinforcement Learning under Partial Observation}, author = {Rafieioskouei, Arshia and Hsu, Tzu-Han and Lucas, Matthew and Bonakdarpour, Borzoo}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {102974--103001}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/rafieioskouei26a/rafieioskouei26a.pdf}, url = {https://proceedings.mlr.press/v306/rafieioskouei26a.html}, abstract = {Formal specification is a powerful tool to guide the learning process and provides significant advantages over reward shaping: (1) mathematical rigor; (2) expressiveness to specify objectives and constraints, and (3) the ability to define tactics to achieve objectives. However, these benefits remain largely unexplored in the context of Multi-Agent Reinforcement Learning (MARL). This paper introduces HyPOLE, a novel framework for MARL under partial observability, where learning is guided by the expressive power of the so-called hyperproperties and, in particular, the temporal logic HyperLTL. We integrate Centralized Training for Decentralized Execution (CTDE) techniques with HyPOLE to synthesize decentralized policies, and our evaluation on SMAC, MessySMAC, and WildFire benchmark demonstrates clear advantages over baselines.} }
Endnote
%0 Conference Paper %T HyPOLE: Hyperproperty-Guided Multi-Agent Reinforcement Learning under Partial Observation %A Arshia Rafieioskouei %A Tzu-Han Hsu %A Matthew Lucas %A Borzoo Bonakdarpour %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-rafieioskouei26a %I PMLR %P 102974--103001 %U https://proceedings.mlr.press/v306/rafieioskouei26a.html %V 306 %X Formal specification is a powerful tool to guide the learning process and provides significant advantages over reward shaping: (1) mathematical rigor; (2) expressiveness to specify objectives and constraints, and (3) the ability to define tactics to achieve objectives. However, these benefits remain largely unexplored in the context of Multi-Agent Reinforcement Learning (MARL). This paper introduces HyPOLE, a novel framework for MARL under partial observability, where learning is guided by the expressive power of the so-called hyperproperties and, in particular, the temporal logic HyperLTL. We integrate Centralized Training for Decentralized Execution (CTDE) techniques with HyPOLE to synthesize decentralized policies, and our evaluation on SMAC, MessySMAC, and WildFire benchmark demonstrates clear advantages over baselines.
APA
Rafieioskouei, A., Hsu, T., Lucas, M. & Bonakdarpour, B.. (2026). HyPOLE: Hyperproperty-Guided Multi-Agent Reinforcement Learning under Partial Observation. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:102974-103001 Available from https://proceedings.mlr.press/v306/rafieioskouei26a.html.

Related Material