Learning Task Informed Abstractions

Xiang Fu, Ge Yang, Pulkit Agrawal, Tommi Jaakkola
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:3480-3491, 2021.

Abstract

Current model-based reinforcement learning methods struggle when operating from complex visual scenes due to their inability to prioritize task-relevant features. To mitigate this problem, we propose learning Task Informed Abstractions (TIA) that explicitly separates reward-correlated visual features from distractors. For learning TIA, we introduce the formalism of Task Informed MDP (TiMDP) that is realized by training two models that learn visual features via cooperative reconstruction, but one model is adversarially dissociated from the reward signal. Empirical evaluation shows that TIA leads to significant performance gains over state-of-the-art methods on many visual control tasks where natural and unconstrained visual distractions pose a formidable challenge. Project page: https://xiangfu.co/tia

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-fu21b, title = {Learning Task Informed Abstractions}, author = {Fu, Xiang and Yang, Ge and Agrawal, Pulkit and Jaakkola, Tommi}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {3480--3491}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/fu21b/fu21b.pdf}, url = {https://proceedings.mlr.press/v139/fu21b.html}, abstract = {Current model-based reinforcement learning methods struggle when operating from complex visual scenes due to their inability to prioritize task-relevant features. To mitigate this problem, we propose learning Task Informed Abstractions (TIA) that explicitly separates reward-correlated visual features from distractors. For learning TIA, we introduce the formalism of Task Informed MDP (TiMDP) that is realized by training two models that learn visual features via cooperative reconstruction, but one model is adversarially dissociated from the reward signal. Empirical evaluation shows that TIA leads to significant performance gains over state-of-the-art methods on many visual control tasks where natural and unconstrained visual distractions pose a formidable challenge. Project page: https://xiangfu.co/tia} }
Endnote
%0 Conference Paper %T Learning Task Informed Abstractions %A Xiang Fu %A Ge Yang %A Pulkit Agrawal %A Tommi Jaakkola %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-fu21b %I PMLR %P 3480--3491 %U https://proceedings.mlr.press/v139/fu21b.html %V 139 %X Current model-based reinforcement learning methods struggle when operating from complex visual scenes due to their inability to prioritize task-relevant features. To mitigate this problem, we propose learning Task Informed Abstractions (TIA) that explicitly separates reward-correlated visual features from distractors. For learning TIA, we introduce the formalism of Task Informed MDP (TiMDP) that is realized by training two models that learn visual features via cooperative reconstruction, but one model is adversarially dissociated from the reward signal. Empirical evaluation shows that TIA leads to significant performance gains over state-of-the-art methods on many visual control tasks where natural and unconstrained visual distractions pose a formidable challenge. Project page: https://xiangfu.co/tia
APA
Fu, X., Yang, G., Agrawal, P. & Jaakkola, T.. (2021). Learning Task Informed Abstractions. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:3480-3491 Available from https://proceedings.mlr.press/v139/fu21b.html.

Related Material