Online Skill Discovery using Graph-based Clustering

Jan Hendrik Metzen
Proceedings of the Tenth European Workshop on Reinforcement Learning, PMLR 24:77-88, 2013.

Abstract

We introduce a new online skill discovery method for reinforcement learning in discrete domains. The method is based on the bottleneck principle and identifies skills using a bottom-up hierarchical clustering of the estimated transition graph. In contrast to prior clustering approaches, it can be used incrementally and thus several times during the learning process. Our empirical evaluation shows that “assuming dense local connectivity in the face of uncertainty” can prevent premature identification of skills. Furthermore, we show that the choice of the linkage criterion is crucial for dealing with non-random sampling policies and stochastic environments.

Cite this Paper

BibTeX
@InProceedings{pmlr-v24-metzen12a, title = {Online Skill Discovery using Graph-based Clustering}, author = {Metzen, Jan Hendrik}, booktitle = {Proceedings of the Tenth European Workshop on Reinforcement Learning}, pages = {77--88}, year = {2013}, editor = {Deisenroth, Marc Peter and Szepesvári, Csaba and Peters, Jan}, volume = {24}, series = {Proceedings of Machine Learning Research}, address = {Edinburgh, Scotland}, month = {30 Jun--01 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v24/metzen12a/metzen12a.pdf}, url = {https://proceedings.mlr.press/v24/metzen12a.html}, abstract = {We introduce a new online skill discovery method for reinforcement learning in discrete domains. The method is based on the bottleneck principle and identifies skills using a bottom-up hierarchical clustering of the estimated transition graph. In contrast to prior clustering approaches, it can be used incrementally and thus several times during the learning process. Our empirical evaluation shows that “assuming dense local connectivity in the face of uncertainty” can prevent premature identification of skills. Furthermore, we show that the choice of the linkage criterion is crucial for dealing with non-random sampling policies and stochastic environments.} }
Endnote
%0 Conference Paper %T Online Skill Discovery using Graph-based Clustering %A Jan Hendrik Metzen %B Proceedings of the Tenth European Workshop on Reinforcement Learning %C Proceedings of Machine Learning Research %D 2013 %E Marc Peter Deisenroth %E Csaba Szepesvári %E Jan Peters %F pmlr-v24-metzen12a %I PMLR %P 77--88 %U https://proceedings.mlr.press/v24/metzen12a.html %V 24 %X We introduce a new online skill discovery method for reinforcement learning in discrete domains. The method is based on the bottleneck principle and identifies skills using a bottom-up hierarchical clustering of the estimated transition graph. In contrast to prior clustering approaches, it can be used incrementally and thus several times during the learning process. Our empirical evaluation shows that “assuming dense local connectivity in the face of uncertainty” can prevent premature identification of skills. Furthermore, we show that the choice of the linkage criterion is crucial for dealing with non-random sampling policies and stochastic environments.
RIS
TY - CPAPER TI - Online Skill Discovery using Graph-based Clustering AU - Jan Hendrik Metzen BT - Proceedings of the Tenth European Workshop on Reinforcement Learning DA - 2013/01/12 ED - Marc Peter Deisenroth ED - Csaba Szepesvári ED - Jan Peters ID - pmlr-v24-metzen12a PB - PMLR DP - Proceedings of Machine Learning Research VL - 24 SP - 77 EP - 88 L1 - http://proceedings.mlr.press/v24/metzen12a/metzen12a.pdf UR - https://proceedings.mlr.press/v24/metzen12a.html AB - We introduce a new online skill discovery method for reinforcement learning in discrete domains. The method is based on the bottleneck principle and identifies skills using a bottom-up hierarchical clustering of the estimated transition graph. In contrast to prior clustering approaches, it can be used incrementally and thus several times during the learning process. Our empirical evaluation shows that “assuming dense local connectivity in the face of uncertainty” can prevent premature identification of skills. Furthermore, we show that the choice of the linkage criterion is crucial for dealing with non-random sampling policies and stochastic environments. ER -
APA
Metzen, J.H.. (2013). Online Skill Discovery using Graph-based Clustering. Proceedings of the Tenth European Workshop on Reinforcement Learning, in Proceedings of Machine Learning Research 24:77-88 Available from https://proceedings.mlr.press/v24/metzen12a.html.

Related Material