The Natural Language of Actions

Guy Tennenholtz, Shie Mannor
Proceedings of the 36th International Conference on Machine Learning, PMLR 97:6196-6205, 2019.

Abstract

We introduce Act2Vec, a general framework for learning context-based action representation for Reinforcement Learning. Representing actions in a vector space help reinforcement learning algorithms achieve better performance by grouping similar actions and utilizing relations between different actions. We show how prior knowledge of an environment can be extracted from demonstrations and injected into action vector representations that encode natural compatible behavior. We then use these for augmenting state representations as well as improving function approximation of Q-values. We visualize and test action embeddings in three domains including a drawing task, a high dimensional navigation task, and the large action space domain of StarCraft II.

Cite this Paper

BibTeX
@InProceedings{pmlr-v97-tennenholtz19a, title = {The Natural Language of Actions}, author = {Tennenholtz, Guy and Mannor, Shie}, booktitle = {Proceedings of the 36th International Conference on Machine Learning}, pages = {6196--6205}, year = {2019}, editor = {Chaudhuri, Kamalika and Salakhutdinov, Ruslan}, volume = {97}, series = {Proceedings of Machine Learning Research}, month = {09--15 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v97/tennenholtz19a/tennenholtz19a.pdf}, url = {https://proceedings.mlr.press/v97/tennenholtz19a.html}, abstract = {We introduce Act2Vec, a general framework for learning context-based action representation for Reinforcement Learning. Representing actions in a vector space help reinforcement learning algorithms achieve better performance by grouping similar actions and utilizing relations between different actions. We show how prior knowledge of an environment can be extracted from demonstrations and injected into action vector representations that encode natural compatible behavior. We then use these for augmenting state representations as well as improving function approximation of Q-values. We visualize and test action embeddings in three domains including a drawing task, a high dimensional navigation task, and the large action space domain of StarCraft II.} }
Endnote
%0 Conference Paper %T The Natural Language of Actions %A Guy Tennenholtz %A Shie Mannor %B Proceedings of the 36th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2019 %E Kamalika Chaudhuri %E Ruslan Salakhutdinov %F pmlr-v97-tennenholtz19a %I PMLR %P 6196--6205 %U https://proceedings.mlr.press/v97/tennenholtz19a.html %V 97 %X We introduce Act2Vec, a general framework for learning context-based action representation for Reinforcement Learning. Representing actions in a vector space help reinforcement learning algorithms achieve better performance by grouping similar actions and utilizing relations between different actions. We show how prior knowledge of an environment can be extracted from demonstrations and injected into action vector representations that encode natural compatible behavior. We then use these for augmenting state representations as well as improving function approximation of Q-values. We visualize and test action embeddings in three domains including a drawing task, a high dimensional navigation task, and the large action space domain of StarCraft II.
APA
Tennenholtz, G. & Mannor, S.. (2019). The Natural Language of Actions. Proceedings of the 36th International Conference on Machine Learning, in Proceedings of Machine Learning Research 97:6196-6205 Available from https://proceedings.mlr.press/v97/tennenholtz19a.html.

Related Material