CURL: Contrastive Unsupervised Representations for Reinforcement Learning

Michael Laskin, Aravind Srinivas, Pieter Abbeel
Proceedings of the 37th International Conference on Machine Learning, PMLR 119:5639-5650, 2020.

Abstract

We present CURL: Contrastive Unsupervised Representations for Reinforcement Learning. CURL extracts high-level features from raw pixels using contrastive learning and performs off-policy control on top of the extracted features. CURL outperforms prior pixel-based methods, both model-based and model-free, on complex tasks in the DeepMind Control Suite and Atari Games showing 1.9x and 1.2x performance gains at the 100K environment and interaction steps benchmarks respectively. On the DeepMind Control Suite, CURL is the first image-based algorithm to nearly match the sample-efficiency of methods that use state-based features. Our code is open-sourced and available at https://www.github.com/MishaLaskin/curl.

Cite this Paper

BibTeX
@InProceedings{pmlr-v119-laskin20a, title = {{CURL}: Contrastive Unsupervised Representations for Reinforcement Learning}, author = {Laskin, Michael and Srinivas, Aravind and Abbeel, Pieter}, booktitle = {Proceedings of the 37th International Conference on Machine Learning}, pages = {5639--5650}, year = {2020}, editor = {III, Hal Daumé and Singh, Aarti}, volume = {119}, series = {Proceedings of Machine Learning Research}, month = {13--18 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v119/laskin20a/laskin20a.pdf}, url = {https://proceedings.mlr.press/v119/laskin20a.html}, abstract = {We present CURL: Contrastive Unsupervised Representations for Reinforcement Learning. CURL extracts high-level features from raw pixels using contrastive learning and performs off-policy control on top of the extracted features. CURL outperforms prior pixel-based methods, both model-based and model-free, on complex tasks in the DeepMind Control Suite and Atari Games showing 1.9x and 1.2x performance gains at the 100K environment and interaction steps benchmarks respectively. On the DeepMind Control Suite, CURL is the first image-based algorithm to nearly match the sample-efficiency of methods that use state-based features. Our code is open-sourced and available at https://www.github.com/MishaLaskin/curl.} }
Endnote
%0 Conference Paper %T CURL: Contrastive Unsupervised Representations for Reinforcement Learning %A Michael Laskin %A Aravind Srinivas %A Pieter Abbeel %B Proceedings of the 37th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2020 %E Hal Daumé III %E Aarti Singh %F pmlr-v119-laskin20a %I PMLR %P 5639--5650 %U https://proceedings.mlr.press/v119/laskin20a.html %V 119 %X We present CURL: Contrastive Unsupervised Representations for Reinforcement Learning. CURL extracts high-level features from raw pixels using contrastive learning and performs off-policy control on top of the extracted features. CURL outperforms prior pixel-based methods, both model-based and model-free, on complex tasks in the DeepMind Control Suite and Atari Games showing 1.9x and 1.2x performance gains at the 100K environment and interaction steps benchmarks respectively. On the DeepMind Control Suite, CURL is the first image-based algorithm to nearly match the sample-efficiency of methods that use state-based features. Our code is open-sourced and available at https://www.github.com/MishaLaskin/curl.
APA
Laskin, M., Srinivas, A. & Abbeel, P.. (2020). CURL: Contrastive Unsupervised Representations for Reinforcement Learning. Proceedings of the 37th International Conference on Machine Learning, in Proceedings of Machine Learning Research 119:5639-5650 Available from https://proceedings.mlr.press/v119/laskin20a.html.

Related Material