Laplacian Representations for Decision-Time Planning

Dikshant Shehmar, Matthew Kyle Schlegel, Matthew E. Taylor, Marlos C. Machado
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:110655-110681, 2026.

Abstract

Planning with a learned model remains a key challenge in model-based reinforcement learning (RL). In decision-time planning, state representations are critical as they must support local cost computation while preserving long-horizon structure. In this paper, we show that the Laplacian representation provides an effective latent space for planning by capturing state-space distances at multiple time scales. This representation preserves meaningful distances and naturally decomposes long-horizon problems into subgoals, also mitigating the compounding errors that arise over long prediction horizons. Building on these properties, we introduce ALPS, a hierarchical planning algorithm, and demonstrate that it outperforms commonly used model-free baselines on a selection of offline goal-conditioned RL tasks from OGBench.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-shehmar26a, title = {{L}aplacian Representations for Decision-Time Planning}, author = {Shehmar, Dikshant and Schlegel, Matthew Kyle and Taylor, Matthew E. and C. Machado, Marlos}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {110655--110681}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/shehmar26a/shehmar26a.pdf}, url = {https://proceedings.mlr.press/v306/shehmar26a.html}, abstract = {Planning with a learned model remains a key challenge in model-based reinforcement learning (RL). In decision-time planning, state representations are critical as they must support local cost computation while preserving long-horizon structure. In this paper, we show that the Laplacian representation provides an effective latent space for planning by capturing state-space distances at multiple time scales. This representation preserves meaningful distances and naturally decomposes long-horizon problems into subgoals, also mitigating the compounding errors that arise over long prediction horizons. Building on these properties, we introduce ALPS, a hierarchical planning algorithm, and demonstrate that it outperforms commonly used model-free baselines on a selection of offline goal-conditioned RL tasks from OGBench.} }
Endnote
%0 Conference Paper %T Laplacian Representations for Decision-Time Planning %A Dikshant Shehmar %A Matthew Kyle Schlegel %A Matthew E. Taylor %A Marlos C. Machado %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-shehmar26a %I PMLR %P 110655--110681 %U https://proceedings.mlr.press/v306/shehmar26a.html %V 306 %X Planning with a learned model remains a key challenge in model-based reinforcement learning (RL). In decision-time planning, state representations are critical as they must support local cost computation while preserving long-horizon structure. In this paper, we show that the Laplacian representation provides an effective latent space for planning by capturing state-space distances at multiple time scales. This representation preserves meaningful distances and naturally decomposes long-horizon problems into subgoals, also mitigating the compounding errors that arise over long prediction horizons. Building on these properties, we introduce ALPS, a hierarchical planning algorithm, and demonstrate that it outperforms commonly used model-free baselines on a selection of offline goal-conditioned RL tasks from OGBench.
APA
Shehmar, D., Schlegel, M.K., Taylor, M.E. & C. Machado, M.. (2026). Laplacian Representations for Decision-Time Planning. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:110655-110681 Available from https://proceedings.mlr.press/v306/shehmar26a.html.

Related Material