SMuCo: Reinforcement Learning for Visual Control via Sequential Multi-view Total Correlation

Tong Cheng, Hang Dong, Lu Wang, Bo Qiao, Qingwei Lin, Saravan Rajmohan, Thomas Moscibroda
Proceedings of the Fortieth Conference on Uncertainty in Artificial Intelligence, PMLR 244:698-717, 2024.

Abstract

The advent of abundant image data has catalyzed the advancement of visual control in reinforcement learning (RL) systems, leveraging multiple view- points to capture the same physical states, which could enhance control performance theoretically. However, integrating multi-view data into representation learning remains challenging. In this paper, we introduce SMuCo, an innovative multi-view reinforcement learning algorithm that constructs robust latent representations by optimizing multi- view sequential total correlation. This technique effectively captures task-relevant information and temporal dynamics while filtering out irrelevant data. Our method supports an unlimited number of views and demonstrates superior performance over leading model-free and model-based RL algorithms. Empirical results from the DeepMind Control Suite and the Sapien Basic Manipulation Task confirm SMuCo’s enhanced efficacy, significantly improving task performance across diverse scenarios and views.

Cite this Paper

BibTeX
@InProceedings{pmlr-v244-cheng24a, title = {SMuCo: Reinforcement Learning for Visual Control via Sequential Multi-view Total Correlation}, author = {Cheng, Tong and Dong, Hang and Wang, Lu and Qiao, Bo and Lin, Qingwei and Rajmohan, Saravan and Moscibroda, Thomas}, booktitle = {Proceedings of the Fortieth Conference on Uncertainty in Artificial Intelligence}, pages = {698--717}, year = {2024}, editor = {Kiyavash, Negar and Mooij, Joris M.}, volume = {244}, series = {Proceedings of Machine Learning Research}, month = {15--19 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v244/main/assets/cheng24a/cheng24a.pdf}, url = {https://proceedings.mlr.press/v244/cheng24a.html}, abstract = {The advent of abundant image data has catalyzed the advancement of visual control in reinforcement learning (RL) systems, leveraging multiple view- points to capture the same physical states, which could enhance control performance theoretically. However, integrating multi-view data into representation learning remains challenging. In this paper, we introduce SMuCo, an innovative multi-view reinforcement learning algorithm that constructs robust latent representations by optimizing multi- view sequential total correlation. This technique effectively captures task-relevant information and temporal dynamics while filtering out irrelevant data. Our method supports an unlimited number of views and demonstrates superior performance over leading model-free and model-based RL algorithms. Empirical results from the DeepMind Control Suite and the Sapien Basic Manipulation Task confirm SMuCo’s enhanced efficacy, significantly improving task performance across diverse scenarios and views.} }
Endnote
%0 Conference Paper %T SMuCo: Reinforcement Learning for Visual Control via Sequential Multi-view Total Correlation %A Tong Cheng %A Hang Dong %A Lu Wang %A Bo Qiao %A Qingwei Lin %A Saravan Rajmohan %A Thomas Moscibroda %B Proceedings of the Fortieth Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2024 %E Negar Kiyavash %E Joris M. Mooij %F pmlr-v244-cheng24a %I PMLR %P 698--717 %U https://proceedings.mlr.press/v244/cheng24a.html %V 244 %X The advent of abundant image data has catalyzed the advancement of visual control in reinforcement learning (RL) systems, leveraging multiple view- points to capture the same physical states, which could enhance control performance theoretically. However, integrating multi-view data into representation learning remains challenging. In this paper, we introduce SMuCo, an innovative multi-view reinforcement learning algorithm that constructs robust latent representations by optimizing multi- view sequential total correlation. This technique effectively captures task-relevant information and temporal dynamics while filtering out irrelevant data. Our method supports an unlimited number of views and demonstrates superior performance over leading model-free and model-based RL algorithms. Empirical results from the DeepMind Control Suite and the Sapien Basic Manipulation Task confirm SMuCo’s enhanced efficacy, significantly improving task performance across diverse scenarios and views.
APA
Cheng, T., Dong, H., Wang, L., Qiao, B., Lin, Q., Rajmohan, S. & Moscibroda, T.. (2024). SMuCo: Reinforcement Learning for Visual Control via Sequential Multi-view Total Correlation. Proceedings of the Fortieth Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research 244:698-717 Available from https://proceedings.mlr.press/v244/cheng24a.html.

Related Material