Unsupervised Skill Discovery with Bottleneck Option Learning

Jaekyeom Kim, Seohong Park, Gunhee Kim
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:5572-5582, 2021.

Abstract

Having the ability to acquire inherent skills from environments without any external rewards or supervision like humans is an important problem. We propose a novel unsupervised skill discovery method named Information Bottleneck Option Learning (IBOL). On top of the linearization of environments that promotes more various and distant state transitions, IBOL enables the discovery of diverse skills. It provides the abstraction of the skills learned with the information bottleneck framework for the options with improved stability and encouraged disentanglement. We empirically demonstrate that IBOL outperforms multiple state-of-the-art unsupervised skill discovery methods on the information-theoretic evaluations and downstream tasks in MuJoCo environments, including Ant, HalfCheetah, Hopper and D’Kitty. Our code is available at https://vision.snu.ac.kr/projects/ibol.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-kim21j, title = {Unsupervised Skill Discovery with Bottleneck Option Learning}, author = {Kim, Jaekyeom and Park, Seohong and Kim, Gunhee}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {5572--5582}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/kim21j/kim21j.pdf}, url = {https://proceedings.mlr.press/v139/kim21j.html}, abstract = {Having the ability to acquire inherent skills from environments without any external rewards or supervision like humans is an important problem. We propose a novel unsupervised skill discovery method named Information Bottleneck Option Learning (IBOL). On top of the linearization of environments that promotes more various and distant state transitions, IBOL enables the discovery of diverse skills. It provides the abstraction of the skills learned with the information bottleneck framework for the options with improved stability and encouraged disentanglement. We empirically demonstrate that IBOL outperforms multiple state-of-the-art unsupervised skill discovery methods on the information-theoretic evaluations and downstream tasks in MuJoCo environments, including Ant, HalfCheetah, Hopper and D’Kitty. Our code is available at https://vision.snu.ac.kr/projects/ibol.} }
Endnote
%0 Conference Paper %T Unsupervised Skill Discovery with Bottleneck Option Learning %A Jaekyeom Kim %A Seohong Park %A Gunhee Kim %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-kim21j %I PMLR %P 5572--5582 %U https://proceedings.mlr.press/v139/kim21j.html %V 139 %X Having the ability to acquire inherent skills from environments without any external rewards or supervision like humans is an important problem. We propose a novel unsupervised skill discovery method named Information Bottleneck Option Learning (IBOL). On top of the linearization of environments that promotes more various and distant state transitions, IBOL enables the discovery of diverse skills. It provides the abstraction of the skills learned with the information bottleneck framework for the options with improved stability and encouraged disentanglement. We empirically demonstrate that IBOL outperforms multiple state-of-the-art unsupervised skill discovery methods on the information-theoretic evaluations and downstream tasks in MuJoCo environments, including Ant, HalfCheetah, Hopper and D’Kitty. Our code is available at https://vision.snu.ac.kr/projects/ibol.
APA
Kim, J., Park, S. & Kim, G.. (2021). Unsupervised Skill Discovery with Bottleneck Option Learning. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:5572-5582 Available from https://proceedings.mlr.press/v139/kim21j.html.

Related Material