Seeing the Forest Despite the Trees: Large Scale Spatial-Temporal Decision Making

Mark Crowley, David Poole, John Nelson
Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence, PMLR R7:126-134, 2009.

Abstract

We introduce a challenging real-world planning problem where actions must be taken at each location in a spatial area at each point in time. We use forestry planning as the motivating application. In Large Scale Spatial-Temporal (LSST) planning problems, the state and action spaces are defined as the cross-products of many local state and action spaces spread over a large spatial area such as a city or forest. These problems possess state uncertainty, have complex utility functions involving spatial constraints and we generally must rely on simulations rather than an explicit transition model. We define LSST problems as reinforcement learning problems and present a solution using policy gradients. We compare two different policy formulations: an explicit policy that identifies each location in space and the action to take there; and an abstract policy that defines the proportion of actions to take across all locations in space. We show that the abstract policy is more robust and achieves higher rewards with far fewer parameters than the elementary policy. This abstract policy is also a better fit to the properties that practitioners in LSST problem domains require for such methods to be widely useful.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR7-crowley09a, title = {Seeing the Forest Despite the Trees: Large Scale Spatial-Temporal Decision Making}, author = {Crowley, Mark and Poole, David and Nelson, John}, booktitle = {Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence}, pages = {126--134}, year = {2009}, editor = {Bilmes, Jeff and Ng, Andrew Y.}, volume = {R7}, series = {Proceedings of Machine Learning Research}, month = {18--21 Jun}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r7/main/assets/crowley09a/crowley09a.pdf}, url = {https://proceedings.mlr.press/r7/crowley09a.html}, abstract = {We introduce a challenging real-world planning problem where actions must be taken at each location in a spatial area at each point in time. We use forestry planning as the motivating application. In Large Scale Spatial-Temporal (LSST) planning problems, the state and action spaces are defined as the cross-products of many local state and action spaces spread over a large spatial area such as a city or forest. These problems possess state uncertainty, have complex utility functions involving spatial constraints and we generally must rely on simulations rather than an explicit transition model. We define LSST problems as reinforcement learning problems and present a solution using policy gradients. We compare two different policy formulations: an explicit policy that identifies each location in space and the action to take there; and an abstract policy that defines the proportion of actions to take across all locations in space. We show that the abstract policy is more robust and achieves higher rewards with far fewer parameters than the elementary policy. This abstract policy is also a better fit to the properties that practitioners in LSST problem domains require for such methods to be widely useful.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T Seeing the Forest Despite the Trees: Large Scale Spatial-Temporal Decision Making %A Mark Crowley %A David Poole %A John Nelson %B Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2009 %E Jeff Bilmes %E Andrew Y. Ng %F pmlr-vR7-crowley09a %I PMLR %P 126--134 %U https://proceedings.mlr.press/r7/crowley09a.html %V R7 %X We introduce a challenging real-world planning problem where actions must be taken at each location in a spatial area at each point in time. We use forestry planning as the motivating application. In Large Scale Spatial-Temporal (LSST) planning problems, the state and action spaces are defined as the cross-products of many local state and action spaces spread over a large spatial area such as a city or forest. These problems possess state uncertainty, have complex utility functions involving spatial constraints and we generally must rely on simulations rather than an explicit transition model. We define LSST problems as reinforcement learning problems and present a solution using policy gradients. We compare two different policy formulations: an explicit policy that identifies each location in space and the action to take there; and an abstract policy that defines the proportion of actions to take across all locations in space. We show that the abstract policy is more robust and achieves higher rewards with far fewer parameters than the elementary policy. This abstract policy is also a better fit to the properties that practitioners in LSST problem domains require for such methods to be widely useful. %Z Reissued by PMLR on 04 October 2026.
APA
Crowley, M., Poole, D. & Nelson, J.. (2009). Seeing the Forest Despite the Trees: Large Scale Spatial-Temporal Decision Making. Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R7:126-134 Available from https://proceedings.mlr.press/r7/crowley09a.html. Reissued by PMLR on 04 October 2026.

Related Material