Bayesian Learning of Recursively Factored Environments

Marc Bellemare, Joel Veness, Michael Bowling
Proceedings of the 30th International Conference on Machine Learning, PMLR 28(3):1211-1219, 2013.

Abstract

Model-based reinforcement learning techniques have historically encountered a number of difficulties scaling up to large observation spaces. One promising approach has been to decompose the model learning task into a number of smaller, more manageable sub-problems by factoring the observation space. Typically, many different factorizations are possible, which can make it difficult to select an appropriate factorization without extensive testing. In this paper we introduce the class of recursively decomposable factorizations, and show how exact Bayesian inference can be used to efficiently guarantee predictive performance close to the best factorization in this class. We demonstrate the strength of this approach by presenting a collection of empirical results for 20 different Atari 2600 games.

Cite this Paper

BibTeX
@InProceedings{pmlr-v28-bellemare13, title = {Bayesian Learning of Recursively Factored Environments}, author = {Bellemare, Marc and Veness, Joel and Bowling, Michael}, booktitle = {Proceedings of the 30th International Conference on Machine Learning}, pages = {1211--1219}, year = {2013}, editor = {Dasgupta, Sanjoy and McAllester, David}, volume = {28}, number = {3}, series = {Proceedings of Machine Learning Research}, address = {Atlanta, Georgia, USA}, month = {17--19 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v28/bellemare13.pdf}, url = {https://proceedings.mlr.press/v28/bellemare13.html}, abstract = {Model-based reinforcement learning techniques have historically encountered a number of difficulties scaling up to large observation spaces. One promising approach has been to decompose the model learning task into a number of smaller, more manageable sub-problems by factoring the observation space. Typically, many different factorizations are possible, which can make it difficult to select an appropriate factorization without extensive testing. In this paper we introduce the class of recursively decomposable factorizations, and show how exact Bayesian inference can be used to efficiently guarantee predictive performance close to the best factorization in this class. We demonstrate the strength of this approach by presenting a collection of empirical results for 20 different Atari 2600 games. } }
Endnote
%0 Conference Paper %T Bayesian Learning of Recursively Factored Environments %A Marc Bellemare %A Joel Veness %A Michael Bowling %B Proceedings of the 30th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2013 %E Sanjoy Dasgupta %E David McAllester %F pmlr-v28-bellemare13 %I PMLR %P 1211--1219 %U https://proceedings.mlr.press/v28/bellemare13.html %V 28 %N 3 %X Model-based reinforcement learning techniques have historically encountered a number of difficulties scaling up to large observation spaces. One promising approach has been to decompose the model learning task into a number of smaller, more manageable sub-problems by factoring the observation space. Typically, many different factorizations are possible, which can make it difficult to select an appropriate factorization without extensive testing. In this paper we introduce the class of recursively decomposable factorizations, and show how exact Bayesian inference can be used to efficiently guarantee predictive performance close to the best factorization in this class. We demonstrate the strength of this approach by presenting a collection of empirical results for 20 different Atari 2600 games.
RIS
TY - CPAPER TI - Bayesian Learning of Recursively Factored Environments AU - Marc Bellemare AU - Joel Veness AU - Michael Bowling BT - Proceedings of the 30th International Conference on Machine Learning DA - 2013/05/26 ED - Sanjoy Dasgupta ED - David McAllester ID - pmlr-v28-bellemare13 PB - PMLR DP - Proceedings of Machine Learning Research VL - 28 IS - 3 SP - 1211 EP - 1219 L1 - http://proceedings.mlr.press/v28/bellemare13.pdf UR - https://proceedings.mlr.press/v28/bellemare13.html AB - Model-based reinforcement learning techniques have historically encountered a number of difficulties scaling up to large observation spaces. One promising approach has been to decompose the model learning task into a number of smaller, more manageable sub-problems by factoring the observation space. Typically, many different factorizations are possible, which can make it difficult to select an appropriate factorization without extensive testing. In this paper we introduce the class of recursively decomposable factorizations, and show how exact Bayesian inference can be used to efficiently guarantee predictive performance close to the best factorization in this class. We demonstrate the strength of this approach by presenting a collection of empirical results for 20 different Atari 2600 games. ER -
APA
Bellemare, M., Veness, J. & Bowling, M.. (2013). Bayesian Learning of Recursively Factored Environments. Proceedings of the 30th International Conference on Machine Learning, in Proceedings of Machine Learning Research 28(3):1211-1219 Available from https://proceedings.mlr.press/v28/bellemare13.html.

Related Material