Specification-Guided Reinforcement Learning

Kishor Jothimurugan, Suguman Bansal, Osbert Bastani, Rajeev Alur
Proceedings of the International Conference on Neuro-symbolic Systems, PMLR 288:316-330, 2025.

Abstract

This tutorial explores specification-guided reinforcement learning as an alternative to traditional reward-based approaches, where the design of effective reward functions can be tedious, error-prone, and may not capture complex objectives. We introduce formal logical specifications as a more intuitive and precise way to define agent behavior, focusing on the theoretical guarantees and algorithmic aspects of learning from specifications. We examine both fundamental limitations in infinite-horizon settings and practical approaches for finite-horizon specifications.

Cite this Paper

BibTeX
@InProceedings{pmlr-v288-jothimurugan25a, title = {Specification-Guided Reinforcement Learning}, author = {Jothimurugan, Kishor and Bansal, Suguman and Bastani, Osbert and Alur, Rajeev}, booktitle = {Proceedings of the International Conference on Neuro-symbolic Systems}, pages = {316--330}, year = {2025}, editor = {Pappas, George and Ravikumar, Pradeep and Seshia, Sanjit A.}, volume = {288}, series = {Proceedings of Machine Learning Research}, month = {28--30 May}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v288/main/assets/jothimurugan25a/jothimurugan25a.pdf}, url = {https://proceedings.mlr.press/v288/jothimurugan25a.html}, abstract = {This tutorial explores specification-guided reinforcement learning as an alternative to traditional reward-based approaches, where the design of effective reward functions can be tedious, error-prone, and may not capture complex objectives. We introduce formal logical specifications as a more intuitive and precise way to define agent behavior, focusing on the theoretical guarantees and algorithmic aspects of learning from specifications. We examine both fundamental limitations in infinite-horizon settings and practical approaches for finite-horizon specifications.} }
Endnote
%0 Conference Paper %T Specification-Guided Reinforcement Learning %A Kishor Jothimurugan %A Suguman Bansal %A Osbert Bastani %A Rajeev Alur %B Proceedings of the International Conference on Neuro-symbolic Systems %C Proceedings of Machine Learning Research %D 2025 %E George Pappas %E Pradeep Ravikumar %E Sanjit A. Seshia %F pmlr-v288-jothimurugan25a %I PMLR %P 316--330 %U https://proceedings.mlr.press/v288/jothimurugan25a.html %V 288 %X This tutorial explores specification-guided reinforcement learning as an alternative to traditional reward-based approaches, where the design of effective reward functions can be tedious, error-prone, and may not capture complex objectives. We introduce formal logical specifications as a more intuitive and precise way to define agent behavior, focusing on the theoretical guarantees and algorithmic aspects of learning from specifications. We examine both fundamental limitations in infinite-horizon settings and practical approaches for finite-horizon specifications.
APA
Jothimurugan, K., Bansal, S., Bastani, O. & Alur, R.. (2025). Specification-Guided Reinforcement Learning. Proceedings of the International Conference on Neuro-symbolic Systems, in Proceedings of Machine Learning Research 288:316-330 Available from https://proceedings.mlr.press/v288/jothimurugan25a.html.

Related Material