Learning to Perceive the World Through Control: Empowerment-Based Representation Learning

Mahsa Bastankhah, Sophie Broderick, Benjamin Eysenbach
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:6908-6938, 2026.

Abstract

In many practical reinforcement learning (RL) environments, observations are far higher-dimensional than the variables that matter for control. In this work, we ask: can we learn representations that capture only control-relevant features of the environment? We study this question through the empowerment objective, which maximizes an agent’s influence over the environment and is widely used for unsupervised skill learning. We show that empowerment agents induce two distinct representations — forward and backward — that capture complementary aspects of the state, and both of which are invariant to control-irrelevant features. Thus, empowerment maximization leads agents to learn an implicit, control-centric model of the world. Our analysis highlights the importance of learning representations through interaction rather than from passive datasets: interaction aimed at maximizing control is essential for learning useful invariance properties, a perspective that aligns closely with the causal learning literature.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-bastankhah26a, title = {Learning to Perceive the World Through Control: Empowerment-Based Representation Learning}, author = {Bastankhah, Mahsa and Broderick, Sophie and Eysenbach, Benjamin}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {6908--6938}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/bastankhah26a/bastankhah26a.pdf}, url = {https://proceedings.mlr.press/v306/bastankhah26a.html}, abstract = {In many practical reinforcement learning (RL) environments, observations are far higher-dimensional than the variables that matter for control. In this work, we ask: can we learn representations that capture only control-relevant features of the environment? We study this question through the empowerment objective, which maximizes an agent’s influence over the environment and is widely used for unsupervised skill learning. We show that empowerment agents induce two distinct representations — forward and backward — that capture complementary aspects of the state, and both of which are invariant to control-irrelevant features. Thus, empowerment maximization leads agents to learn an implicit, control-centric model of the world. Our analysis highlights the importance of learning representations through interaction rather than from passive datasets: interaction aimed at maximizing control is essential for learning useful invariance properties, a perspective that aligns closely with the causal learning literature.} }
Endnote
%0 Conference Paper %T Learning to Perceive the World Through Control: Empowerment-Based Representation Learning %A Mahsa Bastankhah %A Sophie Broderick %A Benjamin Eysenbach %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-bastankhah26a %I PMLR %P 6908--6938 %U https://proceedings.mlr.press/v306/bastankhah26a.html %V 306 %X In many practical reinforcement learning (RL) environments, observations are far higher-dimensional than the variables that matter for control. In this work, we ask: can we learn representations that capture only control-relevant features of the environment? We study this question through the empowerment objective, which maximizes an agent’s influence over the environment and is widely used for unsupervised skill learning. We show that empowerment agents induce two distinct representations — forward and backward — that capture complementary aspects of the state, and both of which are invariant to control-irrelevant features. Thus, empowerment maximization leads agents to learn an implicit, control-centric model of the world. Our analysis highlights the importance of learning representations through interaction rather than from passive datasets: interaction aimed at maximizing control is essential for learning useful invariance properties, a perspective that aligns closely with the causal learning literature.
APA
Bastankhah, M., Broderick, S. & Eysenbach, B.. (2026). Learning to Perceive the World Through Control: Empowerment-Based Representation Learning. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:6908-6938 Available from https://proceedings.mlr.press/v306/bastankhah26a.html.

Related Material