RLang: A Declarative Language for Describing Partial World Knowledge to Reinforcement Learning Agents

Rafael Rodriguez-Sanchez, Benjamin Adin Spiegel, Jennifer Wang, Roma Patel, Stefanie Tellex, George Konidaris
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:29161-29178, 2023.

Abstract

We introduce RLang, a domain-specific language (DSL) for communicating domain knowledge to an RL agent. Unlike existing RL DSLs that ground to $\textit{single}$ elements of a decision-making formalism (e.g., the reward function or policy), RLang can specify information about every element of a Markov decision process. We define precise syntax and grounding semantics for RLang, and provide a parser that grounds RLang programs to an algorithm-agnostic $\textit{partial}$ world model and policy that can be exploited by an RL agent. We provide a series of example RLang programs demonstrating how different RL methods can exploit the resulting knowledge, encompassing model-free and model-based tabular algorithms, policy gradient and value-based methods, hierarchical approaches, and deep methods.

Cite this Paper

BibTeX
@InProceedings{pmlr-v202-rodriguez-sanchez23a, title = {{RL}ang: A Declarative Language for Describing Partial World Knowledge to Reinforcement Learning Agents}, author = {Rodriguez-Sanchez, Rafael and Spiegel, Benjamin Adin and Wang, Jennifer and Patel, Roma and Tellex, Stefanie and Konidaris, George}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {29161--29178}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/rodriguez-sanchez23a/rodriguez-sanchez23a.pdf}, url = {https://proceedings.mlr.press/v202/rodriguez-sanchez23a.html}, abstract = {We introduce RLang, a domain-specific language (DSL) for communicating domain knowledge to an RL agent. Unlike existing RL DSLs that ground to $\textit{single}$ elements of a decision-making formalism (e.g., the reward function or policy), RLang can specify information about every element of a Markov decision process. We define precise syntax and grounding semantics for RLang, and provide a parser that grounds RLang programs to an algorithm-agnostic $\textit{partial}$ world model and policy that can be exploited by an RL agent. We provide a series of example RLang programs demonstrating how different RL methods can exploit the resulting knowledge, encompassing model-free and model-based tabular algorithms, policy gradient and value-based methods, hierarchical approaches, and deep methods.} }
Endnote
%0 Conference Paper %T RLang: A Declarative Language for Describing Partial World Knowledge to Reinforcement Learning Agents %A Rafael Rodriguez-Sanchez %A Benjamin Adin Spiegel %A Jennifer Wang %A Roma Patel %A Stefanie Tellex %A George Konidaris %B Proceedings of the 40th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Andreas Krause %E Emma Brunskill %E Kyunghyun Cho %E Barbara Engelhardt %E Sivan Sabato %E Jonathan Scarlett %F pmlr-v202-rodriguez-sanchez23a %I PMLR %P 29161--29178 %U https://proceedings.mlr.press/v202/rodriguez-sanchez23a.html %V 202 %X We introduce RLang, a domain-specific language (DSL) for communicating domain knowledge to an RL agent. Unlike existing RL DSLs that ground to $\textit{single}$ elements of a decision-making formalism (e.g., the reward function or policy), RLang can specify information about every element of a Markov decision process. We define precise syntax and grounding semantics for RLang, and provide a parser that grounds RLang programs to an algorithm-agnostic $\textit{partial}$ world model and policy that can be exploited by an RL agent. We provide a series of example RLang programs demonstrating how different RL methods can exploit the resulting knowledge, encompassing model-free and model-based tabular algorithms, policy gradient and value-based methods, hierarchical approaches, and deep methods.
APA
Rodriguez-Sanchez, R., Spiegel, B.A., Wang, J., Patel, R., Tellex, S. & Konidaris, G.. (2023). RLang: A Declarative Language for Describing Partial World Knowledge to Reinforcement Learning Agents. Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:29161-29178 Available from https://proceedings.mlr.press/v202/rodriguez-sanchez23a.html.

Related Material