Transfer Learning in Sequential Decision Problems: A Hierarchical Bayesian Approach

Aaron Wilson, Alan Fern, Prasad Tadepalli
Proceedings of ICML Workshop on Unsupervised and Transfer Learning, PMLR 27:217-227, 2012.

Abstract

Transfer learning is one way to close the gap between the apparent speed of human learning and the relatively slow pace of learning by machines. Transfer is doubly beneficial in reinforcement learning where the agent not only needs to generalize from sparse experience, but also needs to efficiently explore. In this paper, we show that the hierarchical Bayesian framework can be readily adapted to sequential decision problems and provides a natural formalization of transfer learning. Using our framework, we produce empirical results in a simple colored maze domain and a complex real-time strategy game. The results show that our Hierarchical Bayesian Transfer framework significantly improves learning speed when tasks are hierarchically related.

Cite this Paper

BibTeX
@InProceedings{pmlr-v27-wilson12a, title = {Transfer Learning in Sequential Decision Problems: A Hierarchical Bayesian Approach}, author = {Wilson, Aaron and Fern, Alan and Tadepalli, Prasad}, booktitle = {Proceedings of ICML Workshop on Unsupervised and Transfer Learning}, pages = {217--227}, year = {2012}, editor = {Guyon, Isabelle and Dror, Gideon and Lemaire, Vincent and Taylor, Graham and Silver, Daniel}, volume = {27}, series = {Proceedings of Machine Learning Research}, address = {Bellevue, Washington, USA}, month = {02 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v27/wilson12a/wilson12a.pdf}, url = {https://proceedings.mlr.press/v27/wilson12a.html}, abstract = {Transfer learning is one way to close the gap between the apparent speed of human learning and the relatively slow pace of learning by machines. Transfer is doubly beneficial in reinforcement learning where the agent not only needs to generalize from sparse experience, but also needs to efficiently explore. In this paper, we show that the hierarchical Bayesian framework can be readily adapted to sequential decision problems and provides a natural formalization of transfer learning. Using our framework, we produce empirical results in a simple colored maze domain and a complex real-time strategy game. The results show that our Hierarchical Bayesian Transfer framework significantly improves learning speed when tasks are hierarchically related.} }
Endnote
%0 Conference Paper %T Transfer Learning in Sequential Decision Problems: A Hierarchical Bayesian Approach %A Aaron Wilson %A Alan Fern %A Prasad Tadepalli %B Proceedings of ICML Workshop on Unsupervised and Transfer Learning %C Proceedings of Machine Learning Research %D 2012 %E Isabelle Guyon %E Gideon Dror %E Vincent Lemaire %E Graham Taylor %E Daniel Silver %F pmlr-v27-wilson12a %I PMLR %P 217--227 %U https://proceedings.mlr.press/v27/wilson12a.html %V 27 %X Transfer learning is one way to close the gap between the apparent speed of human learning and the relatively slow pace of learning by machines. Transfer is doubly beneficial in reinforcement learning where the agent not only needs to generalize from sparse experience, but also needs to efficiently explore. In this paper, we show that the hierarchical Bayesian framework can be readily adapted to sequential decision problems and provides a natural formalization of transfer learning. Using our framework, we produce empirical results in a simple colored maze domain and a complex real-time strategy game. The results show that our Hierarchical Bayesian Transfer framework significantly improves learning speed when tasks are hierarchically related.
RIS
TY - CPAPER TI - Transfer Learning in Sequential Decision Problems: A Hierarchical Bayesian Approach AU - Aaron Wilson AU - Alan Fern AU - Prasad Tadepalli BT - Proceedings of ICML Workshop on Unsupervised and Transfer Learning DA - 2012/06/27 ED - Isabelle Guyon ED - Gideon Dror ED - Vincent Lemaire ED - Graham Taylor ED - Daniel Silver ID - pmlr-v27-wilson12a PB - PMLR DP - Proceedings of Machine Learning Research VL - 27 SP - 217 EP - 227 L1 - http://proceedings.mlr.press/v27/wilson12a/wilson12a.pdf UR - https://proceedings.mlr.press/v27/wilson12a.html AB - Transfer learning is one way to close the gap between the apparent speed of human learning and the relatively slow pace of learning by machines. Transfer is doubly beneficial in reinforcement learning where the agent not only needs to generalize from sparse experience, but also needs to efficiently explore. In this paper, we show that the hierarchical Bayesian framework can be readily adapted to sequential decision problems and provides a natural formalization of transfer learning. Using our framework, we produce empirical results in a simple colored maze domain and a complex real-time strategy game. The results show that our Hierarchical Bayesian Transfer framework significantly improves learning speed when tasks are hierarchically related. ER -
APA
Wilson, A., Fern, A. & Tadepalli, P.. (2012). Transfer Learning in Sequential Decision Problems: A Hierarchical Bayesian Approach. Proceedings of ICML Workshop on Unsupervised and Transfer Learning, in Proceedings of Machine Learning Research 27:217-227 Available from https://proceedings.mlr.press/v27/wilson12a.html.

Related Material