Language-guided Skill Learning with Temporal Variational Inference

Haotian Fu, Pratyusha Sharma, Elias Stengel-Eskin, George Konidaris, Nicolas Le Roux, Marc-Alexandre Côté, Xingdi Yuan
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:14135-14156, 2024.

Abstract

We present an algorithm for skill discovery from expert demonstrations. The algorithm first utilizes Large Language Models (LLMs) to propose an initial segmentation of the trajectories. Following that, a hierarchical variational inference framework incorporates the LLM-generated segmentation information to discover reusable skills by merging trajectory segments. To further control the trade-off between compression and reusability, we introduce a novel auxiliary objective based on the Minimum Description Length principle that helps guide this skill discovery process. Our results demonstrate that agents equipped with our method are able to discover skills that help accelerate learning and outperform baseline skill learning approaches on new long-horizon tasks in BabyAI, a grid world navigation environment, as well as ALFRED, a household simulation environment.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-fu24e, title = {Language-guided Skill Learning with Temporal Variational Inference}, author = {Fu, Haotian and Sharma, Pratyusha and Stengel-Eskin, Elias and Konidaris, George and Le Roux, Nicolas and C\^{o}t\'{e}, Marc-Alexandre and Yuan, Xingdi}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {14135--14156}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/fu24e/fu24e.pdf}, url = {https://proceedings.mlr.press/v235/fu24e.html}, abstract = {We present an algorithm for skill discovery from expert demonstrations. The algorithm first utilizes Large Language Models (LLMs) to propose an initial segmentation of the trajectories. Following that, a hierarchical variational inference framework incorporates the LLM-generated segmentation information to discover reusable skills by merging trajectory segments. To further control the trade-off between compression and reusability, we introduce a novel auxiliary objective based on the Minimum Description Length principle that helps guide this skill discovery process. Our results demonstrate that agents equipped with our method are able to discover skills that help accelerate learning and outperform baseline skill learning approaches on new long-horizon tasks in BabyAI, a grid world navigation environment, as well as ALFRED, a household simulation environment.} }
Endnote
%0 Conference Paper %T Language-guided Skill Learning with Temporal Variational Inference %A Haotian Fu %A Pratyusha Sharma %A Elias Stengel-Eskin %A George Konidaris %A Nicolas Le Roux %A Marc-Alexandre Côté %A Xingdi Yuan %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-fu24e %I PMLR %P 14135--14156 %U https://proceedings.mlr.press/v235/fu24e.html %V 235 %X We present an algorithm for skill discovery from expert demonstrations. The algorithm first utilizes Large Language Models (LLMs) to propose an initial segmentation of the trajectories. Following that, a hierarchical variational inference framework incorporates the LLM-generated segmentation information to discover reusable skills by merging trajectory segments. To further control the trade-off between compression and reusability, we introduce a novel auxiliary objective based on the Minimum Description Length principle that helps guide this skill discovery process. Our results demonstrate that agents equipped with our method are able to discover skills that help accelerate learning and outperform baseline skill learning approaches on new long-horizon tasks in BabyAI, a grid world navigation environment, as well as ALFRED, a household simulation environment.
APA
Fu, H., Sharma, P., Stengel-Eskin, E., Konidaris, G., Le Roux, N., Côté, M. & Yuan, X.. (2024). Language-guided Skill Learning with Temporal Variational Inference. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:14135-14156 Available from https://proceedings.mlr.press/v235/fu24e.html.

Related Material