Discovering Object-Centric Generalized Value Functions From Pixels

Somjit Nath, Gopeshh Subbaraj, Khimya Khetarpal, Samira Ebrahimi Kahou
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:25755-25768, 2023.

Abstract

Deep Reinforcement Learning has shown significant progress in extracting useful representations from high-dimensional inputs albeit using hand-crafted auxiliary tasks and pseudo rewards. Automatically learning such representations in an object-centric manner geared towards control and fast adaptation remains an open research problem. In this paper, we introduce a method that tries to discover meaningful features from objects, translating them to temporally coherent ‘question’ functions and leveraging the subsequent learned general value functions for control. We compare our approach with state-of-the-art techniques alongside other ablations and show competitive performance in both stationary and non-stationary settings. Finally, we also investigate the discovered general value functions and through qualitative analysis show that the learned representations are not only interpretable but also, centered around objects that are invariant to changes across tasks facilitating fast adaptation.

Cite this Paper

BibTeX
@InProceedings{pmlr-v202-nath23a, title = {Discovering Object-Centric Generalized Value Functions From Pixels}, author = {Nath, Somjit and Subbaraj, Gopeshh and Khetarpal, Khimya and Kahou, Samira Ebrahimi}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {25755--25768}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/nath23a/nath23a.pdf}, url = {https://proceedings.mlr.press/v202/nath23a.html}, abstract = {Deep Reinforcement Learning has shown significant progress in extracting useful representations from high-dimensional inputs albeit using hand-crafted auxiliary tasks and pseudo rewards. Automatically learning such representations in an object-centric manner geared towards control and fast adaptation remains an open research problem. In this paper, we introduce a method that tries to discover meaningful features from objects, translating them to temporally coherent ‘question’ functions and leveraging the subsequent learned general value functions for control. We compare our approach with state-of-the-art techniques alongside other ablations and show competitive performance in both stationary and non-stationary settings. Finally, we also investigate the discovered general value functions and through qualitative analysis show that the learned representations are not only interpretable but also, centered around objects that are invariant to changes across tasks facilitating fast adaptation.} }
Endnote
%0 Conference Paper %T Discovering Object-Centric Generalized Value Functions From Pixels %A Somjit Nath %A Gopeshh Subbaraj %A Khimya Khetarpal %A Samira Ebrahimi Kahou %B Proceedings of the 40th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Andreas Krause %E Emma Brunskill %E Kyunghyun Cho %E Barbara Engelhardt %E Sivan Sabato %E Jonathan Scarlett %F pmlr-v202-nath23a %I PMLR %P 25755--25768 %U https://proceedings.mlr.press/v202/nath23a.html %V 202 %X Deep Reinforcement Learning has shown significant progress in extracting useful representations from high-dimensional inputs albeit using hand-crafted auxiliary tasks and pseudo rewards. Automatically learning such representations in an object-centric manner geared towards control and fast adaptation remains an open research problem. In this paper, we introduce a method that tries to discover meaningful features from objects, translating them to temporally coherent ‘question’ functions and leveraging the subsequent learned general value functions for control. We compare our approach with state-of-the-art techniques alongside other ablations and show competitive performance in both stationary and non-stationary settings. Finally, we also investigate the discovered general value functions and through qualitative analysis show that the learned representations are not only interpretable but also, centered around objects that are invariant to changes across tasks facilitating fast adaptation.
APA
Nath, S., Subbaraj, G., Khetarpal, K. & Kahou, S.E.. (2023). Discovering Object-Centric Generalized Value Functions From Pixels. Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:25755-25768 Available from https://proceedings.mlr.press/v202/nath23a.html.

Related Material