Latent Goal Analysis for Dimension Reduction in Reinforcement Learning

Matthias Rolf, Minoru Asada
Proceedings of The 4th Workshop on Machine Learning for Interactive Systems at ICML 2015, PMLR 43:26-30, 2015.

Abstract

In contrast to reinforcement learning, adaptive control formulations [Nguyen-Tuong and Peters, 2011] already come with expressive and typically low-dimensional goal and task representations, which have been generally considered more expressive than the RL setting [Kaelbling et al., 1996]. Goal and actual values in motor control define a relation similar [Rolf and Steil, 2014] to actual and target outputs in classical supervised learning settings by providing “directional information” in contrast to a mere “magnitude of an error” in reinforcement learning [Barto, 1994]. Recent work [Rolf and Asada, 2014] however showed that these two problem formulations can be transformed into each other. Hence, highly descriptive task representations can be extracted out of reinforcement learning problems by transforming them into adaptive control problems. After introducing the method called Latent Goal Analysis, we discuss the possible application of this approach as dimension reduction technique in reinforcement learning. Experimental results in a web recommender scenario confirm the potential of this technique.

Cite this Paper

BibTeX
@InProceedings{pmlr-v43-rolf15, title = {Latent Goal Analysis for Dimension Reduction in Reinforcement Learning}, author = {Rolf, Matthias and Asada, Minoru}, booktitle = {Proceedings of The 4th Workshop on Machine Learning for Interactive Systems at ICML 2015}, pages = {26--30}, year = {2015}, editor = {Cuayáhuitl, Heriberto and Dethlefs, Nina and Frommberger, Lutz and Van Otterlo, Martijn and Pietquin, Olivier}, volume = {43}, series = {Proceedings of Machine Learning Research}, address = {Lille, France}, month = {11 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v43/rolf15.pdf}, url = {https://proceedings.mlr.press/v43/rolf15.html}, abstract = {In contrast to reinforcement learning, adaptive control formulations [Nguyen-Tuong and Peters, 2011] already come with expressive and typically low-dimensional goal and task representations, which have been generally considered more expressive than the RL setting [Kaelbling et al., 1996]. Goal and actual values in motor control define a relation similar [Rolf and Steil, 2014] to actual and target outputs in classical supervised learning settings by providing “directional information” in contrast to a mere “magnitude of an error” in reinforcement learning [Barto, 1994]. Recent work [Rolf and Asada, 2014] however showed that these two problem formulations can be transformed into each other. Hence, highly descriptive task representations can be extracted out of reinforcement learning problems by transforming them into adaptive control problems. After introducing the method called Latent Goal Analysis, we discuss the possible application of this approach as dimension reduction technique in reinforcement learning. Experimental results in a web recommender scenario confirm the potential of this technique.} }
Endnote
%0 Conference Paper %T Latent Goal Analysis for Dimension Reduction in Reinforcement Learning %A Matthias Rolf %A Minoru Asada %B Proceedings of The 4th Workshop on Machine Learning for Interactive Systems at ICML 2015 %C Proceedings of Machine Learning Research %D 2015 %E Heriberto Cuayáhuitl %E Nina Dethlefs %E Lutz Frommberger %E Martijn Van Otterlo %E Olivier Pietquin %F pmlr-v43-rolf15 %I PMLR %P 26--30 %U https://proceedings.mlr.press/v43/rolf15.html %V 43 %X In contrast to reinforcement learning, adaptive control formulations [Nguyen-Tuong and Peters, 2011] already come with expressive and typically low-dimensional goal and task representations, which have been generally considered more expressive than the RL setting [Kaelbling et al., 1996]. Goal and actual values in motor control define a relation similar [Rolf and Steil, 2014] to actual and target outputs in classical supervised learning settings by providing “directional information” in contrast to a mere “magnitude of an error” in reinforcement learning [Barto, 1994]. Recent work [Rolf and Asada, 2014] however showed that these two problem formulations can be transformed into each other. Hence, highly descriptive task representations can be extracted out of reinforcement learning problems by transforming them into adaptive control problems. After introducing the method called Latent Goal Analysis, we discuss the possible application of this approach as dimension reduction technique in reinforcement learning. Experimental results in a web recommender scenario confirm the potential of this technique.
RIS
TY - CPAPER TI - Latent Goal Analysis for Dimension Reduction in Reinforcement Learning AU - Matthias Rolf AU - Minoru Asada BT - Proceedings of The 4th Workshop on Machine Learning for Interactive Systems at ICML 2015 DA - 2015/06/18 ED - Heriberto Cuayáhuitl ED - Nina Dethlefs ED - Lutz Frommberger ED - Martijn Van Otterlo ED - Olivier Pietquin ID - pmlr-v43-rolf15 PB - PMLR DP - Proceedings of Machine Learning Research VL - 43 SP - 26 EP - 30 L1 - http://proceedings.mlr.press/v43/rolf15.pdf UR - https://proceedings.mlr.press/v43/rolf15.html AB - In contrast to reinforcement learning, adaptive control formulations [Nguyen-Tuong and Peters, 2011] already come with expressive and typically low-dimensional goal and task representations, which have been generally considered more expressive than the RL setting [Kaelbling et al., 1996]. Goal and actual values in motor control define a relation similar [Rolf and Steil, 2014] to actual and target outputs in classical supervised learning settings by providing “directional information” in contrast to a mere “magnitude of an error” in reinforcement learning [Barto, 1994]. Recent work [Rolf and Asada, 2014] however showed that these two problem formulations can be transformed into each other. Hence, highly descriptive task representations can be extracted out of reinforcement learning problems by transforming them into adaptive control problems. After introducing the method called Latent Goal Analysis, we discuss the possible application of this approach as dimension reduction technique in reinforcement learning. Experimental results in a web recommender scenario confirm the potential of this technique. ER -
APA
Rolf, M. & Asada, M.. (2015). Latent Goal Analysis for Dimension Reduction in Reinforcement Learning. Proceedings of The 4th Workshop on Machine Learning for Interactive Systems at ICML 2015, in Proceedings of Machine Learning Research 43:26-30 Available from https://proceedings.mlr.press/v43/rolf15.html.

Related Material