CAMPs: Learning Context-Specific Abstractions for Efficient Planning in Factored MDPs

Rohan Chitnis, Tom Silver, Beomjoon Kim, Leslie Kaelbling, Tomas Lozano-Perez
Proceedings of the 2020 Conference on Robot Learning, PMLR 155:64-79, 2021.

Abstract

Meta-planning, or learning to guide planning from experience, is a promising approach to improving the computational cost of planning. A general meta-planning strategy is to learn to impose constraints on the states considered and actions taken by the agent. We observe that (1) imposing a constraint can induce context-specific independences that render some aspects of the domain irrelevant, and (2) an agent can take advantage of this fact by imposing constraints on its own behavior. These observations lead us to propose the context-specific abstract Markov decision process (CAMP), an abstraction of a factored MDP that affords efficient planning. We then describe how to learn constraints to impose so the CAMP optimizes a trade-off between rewards and computational cost. Our experiments consider five planners across four domains, including robotic navigation among movable obstacles (NAMO), robotic task and motion planning for sequential manipulation, and classical planning. We find planning with learned CAMPs to consistently outperform baselines, including Stilman’s NAMO-specific algorithm. Video: https://youtu.be/wTXt6djcAd4 Code: https://git.io/JTnf6

Cite this Paper

BibTeX
@InProceedings{pmlr-v155-chitnis21a, title = {CAMPs: Learning Context-Specific Abstractions for Efficient Planning in Factored MDPs}, author = {Chitnis, Rohan and Silver, Tom and Kim, Beomjoon and Kaelbling, Leslie and Lozano-Perez, Tomas}, booktitle = {Proceedings of the 2020 Conference on Robot Learning}, pages = {64--79}, year = {2021}, editor = {Kober, Jens and Ramos, Fabio and Tomlin, Claire}, volume = {155}, series = {Proceedings of Machine Learning Research}, month = {16--18 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v155/chitnis21a/chitnis21a.pdf}, url = {https://proceedings.mlr.press/v155/chitnis21a.html}, abstract = {Meta-planning, or learning to guide planning from experience, is a promising approach to improving the computational cost of planning. A general meta-planning strategy is to learn to impose constraints on the states considered and actions taken by the agent. We observe that (1) imposing a constraint can induce context-specific independences that render some aspects of the domain irrelevant, and (2) an agent can take advantage of this fact by imposing constraints on its own behavior. These observations lead us to propose the context-specific abstract Markov decision process (CAMP), an abstraction of a factored MDP that affords efficient planning. We then describe how to learn constraints to impose so the CAMP optimizes a trade-off between rewards and computational cost. Our experiments consider five planners across four domains, including robotic navigation among movable obstacles (NAMO), robotic task and motion planning for sequential manipulation, and classical planning. We find planning with learned CAMPs to consistently outperform baselines, including Stilman’s NAMO-specific algorithm. Video: https://youtu.be/wTXt6djcAd4 Code: https://git.io/JTnf6} }
Endnote
%0 Conference Paper %T CAMPs: Learning Context-Specific Abstractions for Efficient Planning in Factored MDPs %A Rohan Chitnis %A Tom Silver %A Beomjoon Kim %A Leslie Kaelbling %A Tomas Lozano-Perez %B Proceedings of the 2020 Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2021 %E Jens Kober %E Fabio Ramos %E Claire Tomlin %F pmlr-v155-chitnis21a %I PMLR %P 64--79 %U https://proceedings.mlr.press/v155/chitnis21a.html %V 155 %X Meta-planning, or learning to guide planning from experience, is a promising approach to improving the computational cost of planning. A general meta-planning strategy is to learn to impose constraints on the states considered and actions taken by the agent. We observe that (1) imposing a constraint can induce context-specific independences that render some aspects of the domain irrelevant, and (2) an agent can take advantage of this fact by imposing constraints on its own behavior. These observations lead us to propose the context-specific abstract Markov decision process (CAMP), an abstraction of a factored MDP that affords efficient planning. We then describe how to learn constraints to impose so the CAMP optimizes a trade-off between rewards and computational cost. Our experiments consider five planners across four domains, including robotic navigation among movable obstacles (NAMO), robotic task and motion planning for sequential manipulation, and classical planning. We find planning with learned CAMPs to consistently outperform baselines, including Stilman’s NAMO-specific algorithm. Video: https://youtu.be/wTXt6djcAd4 Code: https://git.io/JTnf6
APA
Chitnis, R., Silver, T., Kim, B., Kaelbling, L. & Lozano-Perez, T.. (2021). CAMPs: Learning Context-Specific Abstractions for Efficient Planning in Factored MDPs. Proceedings of the 2020 Conference on Robot Learning, in Proceedings of Machine Learning Research 155:64-79 Available from https://proceedings.mlr.press/v155/chitnis21a.html.

Related Material