Mining Causal Signal Temporal Logic Formulas for Efficient Reinforcement Learning with Temporally Extended Tasks

Hadi Partovi Aria, Zhe Xu
Proceedings of the International Conference on Neuro-symbolic Systems, PMLR 288:524-542, 2025.

Abstract

Reinforcement Learning (RL) has emerged as a powerful paradigm for solving sequential decision-making problems. However, traditional RL methods often lack an understanding of the causal mechanisms that govern the dynamics of an environment. This limitation results in inefficiencies, challenges in generalization, and reduced interpretability. To address these challenges, we propose Signal Temporal Logic Causal Inference RL (STL-CIRL), a framework that mines interpretable causal specifications through Signal Temporal Logic and reinforcement learning, using counterexample-guided refinement to jointly optimize policies and causal formulas. We compare the performance of agents leveraging explicit causal knowledge with those relying solely on traditional RL approaches. Our results demonstrate the potential of causal reasoning to enhance the efficiency and robustness of RL for complex tasks.

Cite this Paper

BibTeX
@InProceedings{pmlr-v288-partovi-aria25a, title = {Mining Causal Signal Temporal Logic Formulas for Efficient Reinforcement Learning with Temporally Extended Tasks}, author = {Partovi Aria, Hadi and Xu, Zhe}, booktitle = {Proceedings of the International Conference on Neuro-symbolic Systems}, pages = {524--542}, year = {2025}, editor = {Pappas, George and Ravikumar, Pradeep and Seshia, Sanjit A.}, volume = {288}, series = {Proceedings of Machine Learning Research}, month = {28--30 May}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v288/main/assets/partovi-aria25a/partovi-aria25a.pdf}, url = {https://proceedings.mlr.press/v288/partovi-aria25a.html}, abstract = {Reinforcement Learning (RL) has emerged as a powerful paradigm for solving sequential decision-making problems. However, traditional RL methods often lack an understanding of the causal mechanisms that govern the dynamics of an environment. This limitation results in inefficiencies, challenges in generalization, and reduced interpretability. To address these challenges, we propose Signal Temporal Logic Causal Inference RL (STL-CIRL), a framework that mines interpretable causal specifications through Signal Temporal Logic and reinforcement learning, using counterexample-guided refinement to jointly optimize policies and causal formulas. We compare the performance of agents leveraging explicit causal knowledge with those relying solely on traditional RL approaches. Our results demonstrate the potential of causal reasoning to enhance the efficiency and robustness of RL for complex tasks.} }
Endnote
%0 Conference Paper %T Mining Causal Signal Temporal Logic Formulas for Efficient Reinforcement Learning with Temporally Extended Tasks %A Hadi Partovi Aria %A Zhe Xu %B Proceedings of the International Conference on Neuro-symbolic Systems %C Proceedings of Machine Learning Research %D 2025 %E George Pappas %E Pradeep Ravikumar %E Sanjit A. Seshia %F pmlr-v288-partovi-aria25a %I PMLR %P 524--542 %U https://proceedings.mlr.press/v288/partovi-aria25a.html %V 288 %X Reinforcement Learning (RL) has emerged as a powerful paradigm for solving sequential decision-making problems. However, traditional RL methods often lack an understanding of the causal mechanisms that govern the dynamics of an environment. This limitation results in inefficiencies, challenges in generalization, and reduced interpretability. To address these challenges, we propose Signal Temporal Logic Causal Inference RL (STL-CIRL), a framework that mines interpretable causal specifications through Signal Temporal Logic and reinforcement learning, using counterexample-guided refinement to jointly optimize policies and causal formulas. We compare the performance of agents leveraging explicit causal knowledge with those relying solely on traditional RL approaches. Our results demonstrate the potential of causal reasoning to enhance the efficiency and robustness of RL for complex tasks.
APA
Partovi Aria, H. & Xu, Z.. (2025). Mining Causal Signal Temporal Logic Formulas for Efficient Reinforcement Learning with Temporally Extended Tasks. Proceedings of the International Conference on Neuro-symbolic Systems, in Proceedings of Machine Learning Research 288:524-542 Available from https://proceedings.mlr.press/v288/partovi-aria25a.html.

Related Material