STLGame: Signal Temporal Logic Games in Adversarial Multi-Agent Systems

Shuo Yang, Hongrui Zheng, Cristian-Ioan Vasile, George Pappas, Rahul Mangharam
Proceedings of the 7th Annual Learning for Dynamics \& Control Conference, PMLR 283:1102-1114, 2025.

Abstract

We study how to synthesize a robust and safe policy for autonomous systems under signal temporal logic (STL) tasks in adversarial settings against unknown dynamic agents. To ensure the worstcase STL satisfaction, we propose STLGame, a framework that models the multi-agent system as a two-player zero-sum game, where the ego agents try to maximize the STL satisfaction and other agents minimize it. STLGame aims to find a Nash equilibrium policy profile, which is the best case in terms of robustness against unseen opponent policies, by using the fictitious self-play (FSP) framework. FSP iteratively converges to a Nash profile, even in games set in continuous stateaction spaces. We propose a gradient-based method with differentiable STL formulas, which is crucial in continuous settings to approximate the best responses at each iteration of FSP. We show this key aspect experimentally by comparing with reinforcement learning-based methods to find the best response. Experiments on two standard dynamical system benchmarks, Ackermann steering vehicles and autonomous drones, demonstrate that our converged policy is almost unexploitable and robust to various unseen opponents’ policies. All code and additional experimental results can be found on our project website: https://sites.google.com/view/stlgame

Cite this Paper

BibTeX
@InProceedings{pmlr-v283-yang25a, title = {STLGame: Signal Temporal Logic Games in Adversarial Multi-Agent Systems}, author = {Yang, Shuo and Zheng, Hongrui and Vasile, Cristian-Ioan and Pappas, George and Mangharam, Rahul}, booktitle = {Proceedings of the 7th Annual Learning for Dynamics \& Control Conference}, pages = {1102--1114}, year = {2025}, editor = {Ozay, Necmiye and Balzano, Laura and Panagou, Dimitra and Abate, Alessandro}, volume = {283}, series = {Proceedings of Machine Learning Research}, month = {04--06 Jun}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v283/main/assets/yang25a/yang25a.pdf}, url = {https://proceedings.mlr.press/v283/yang25a.html}, abstract = {We study how to synthesize a robust and safe policy for autonomous systems under signal temporal logic (STL) tasks in adversarial settings against unknown dynamic agents. To ensure the worstcase STL satisfaction, we propose STLGame, a framework that models the multi-agent system as a two-player zero-sum game, where the ego agents try to maximize the STL satisfaction and other agents minimize it. STLGame aims to find a Nash equilibrium policy profile, which is the best case in terms of robustness against unseen opponent policies, by using the fictitious self-play (FSP) framework. FSP iteratively converges to a Nash profile, even in games set in continuous stateaction spaces. We propose a gradient-based method with differentiable STL formulas, which is crucial in continuous settings to approximate the best responses at each iteration of FSP. We show this key aspect experimentally by comparing with reinforcement learning-based methods to find the best response. Experiments on two standard dynamical system benchmarks, Ackermann steering vehicles and autonomous drones, demonstrate that our converged policy is almost unexploitable and robust to various unseen opponents’ policies. All code and additional experimental results can be found on our project website: https://sites.google.com/view/stlgame} }
Endnote
%0 Conference Paper %T STLGame: Signal Temporal Logic Games in Adversarial Multi-Agent Systems %A Shuo Yang %A Hongrui Zheng %A Cristian-Ioan Vasile %A George Pappas %A Rahul Mangharam %B Proceedings of the 7th Annual Learning for Dynamics \& Control Conference %C Proceedings of Machine Learning Research %D 2025 %E Necmiye Ozay %E Laura Balzano %E Dimitra Panagou %E Alessandro Abate %F pmlr-v283-yang25a %I PMLR %P 1102--1114 %U https://proceedings.mlr.press/v283/yang25a.html %V 283 %X We study how to synthesize a robust and safe policy for autonomous systems under signal temporal logic (STL) tasks in adversarial settings against unknown dynamic agents. To ensure the worstcase STL satisfaction, we propose STLGame, a framework that models the multi-agent system as a two-player zero-sum game, where the ego agents try to maximize the STL satisfaction and other agents minimize it. STLGame aims to find a Nash equilibrium policy profile, which is the best case in terms of robustness against unseen opponent policies, by using the fictitious self-play (FSP) framework. FSP iteratively converges to a Nash profile, even in games set in continuous stateaction spaces. We propose a gradient-based method with differentiable STL formulas, which is crucial in continuous settings to approximate the best responses at each iteration of FSP. We show this key aspect experimentally by comparing with reinforcement learning-based methods to find the best response. Experiments on two standard dynamical system benchmarks, Ackermann steering vehicles and autonomous drones, demonstrate that our converged policy is almost unexploitable and robust to various unseen opponents’ policies. All code and additional experimental results can be found on our project website: https://sites.google.com/view/stlgame
APA
Yang, S., Zheng, H., Vasile, C., Pappas, G. & Mangharam, R.. (2025). STLGame: Signal Temporal Logic Games in Adversarial Multi-Agent Systems. Proceedings of the 7th Annual Learning for Dynamics \& Control Conference, in Proceedings of Machine Learning Research 283:1102-1114 Available from https://proceedings.mlr.press/v283/yang25a.html.

Related Material