Planning under Uncertainty with Weighted State Scenarios

Erwin Walraven Delft University of Technology, Matthijs Spaan Delft University of Technology
Proceedings of the 31st Conference on Uncertainty in Artificial Intelligence, PMLR R13:169-178, 2015.

Abstract

Decision making under uncertainty requires reasoning about uncertain events that may occur in the future. In many planning domains external factors are hard to model using a compact Markovian state. However, multiple consecutive state observations received from an environment may be correlated over time, which can be exploited during planning. In this paper we propose a scenario representation which enables agents to reason about sequences of future states. We show how weights can be assigned to scenarios, representing the likelihood that scenarios predict future states. Furthermore, we present a model based on a Partially Observable Markov Decision Process (POMDP), which can be used to reason about state scenarios during planning. In experiments we show how our scenario representation and POMDP model can be applied in the context of smart grids and stock markets. In both domains our model outperforms other methods that do not account for long term correlations.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR13-technology15a, title = {Planning under Uncertainty with Weighted State Scenarios}, author = {Technology, Erwin Walraven Delft University of and Technology, Matthijs Spaan Delft University of}, booktitle = {Proceedings of the 31st Conference on Uncertainty in Artificial Intelligence}, pages = {169--178}, year = {2015}, editor = {Meila, Marina and Heskes, Tom}, volume = {R13}, series = {Proceedings of Machine Learning Research}, month = {12--16 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r13/main/assets/technology15a/technology15a.pdf}, url = {https://proceedings.mlr.press/r13/technology15a.html}, abstract = {Decision making under uncertainty requires reasoning about uncertain events that may occur in the future. In many planning domains external factors are hard to model using a compact Markovian state. However, multiple consecutive state observations received from an environment may be correlated over time, which can be exploited during planning. In this paper we propose a scenario representation which enables agents to reason about sequences of future states. We show how weights can be assigned to scenarios, representing the likelihood that scenarios predict future states. Furthermore, we present a model based on a Partially Observable Markov Decision Process (POMDP), which can be used to reason about state scenarios during planning. In experiments we show how our scenario representation and POMDP model can be applied in the context of smart grids and stock markets. In both domains our model outperforms other methods that do not account for long term correlations.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T Planning under Uncertainty with Weighted State Scenarios %A Erwin Walraven Delft University of Technology %A Matthijs Spaan Delft University of Technology %B Proceedings of the 31st Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2015 %E Marina Meila %E Tom Heskes %F pmlr-vR13-technology15a %I PMLR %P 169--178 %U https://proceedings.mlr.press/r13/technology15a.html %V R13 %X Decision making under uncertainty requires reasoning about uncertain events that may occur in the future. In many planning domains external factors are hard to model using a compact Markovian state. However, multiple consecutive state observations received from an environment may be correlated over time, which can be exploited during planning. In this paper we propose a scenario representation which enables agents to reason about sequences of future states. We show how weights can be assigned to scenarios, representing the likelihood that scenarios predict future states. Furthermore, we present a model based on a Partially Observable Markov Decision Process (POMDP), which can be used to reason about state scenarios during planning. In experiments we show how our scenario representation and POMDP model can be applied in the context of smart grids and stock markets. In both domains our model outperforms other methods that do not account for long term correlations. %Z Reissued by PMLR on 04 October 2026.
APA
Technology, E.W.D.U.o. & Technology, M.S.D.U.o.. (2015). Planning under Uncertainty with Weighted State Scenarios. Proceedings of the 31st Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R13:169-178 Available from https://proceedings.mlr.press/r13/technology15a.html. Reissued by PMLR on 04 October 2026.

Related Material