Approximate Control for Continuous-Time POMDPs

Yannick Eich, Bastian Alt, Heinz Koeppl
Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, PMLR 238:3160-3168, 2024.

Abstract

This work proposes a decision-making framework for partially observable systems in continuous time with discrete state and action spaces. As optimal decision-making becomes intractable for large state spaces we employ approximation methods for the filtering and the control problem that scale well with an increasing number of states. Specifically, we approximate the high-dimensional filtering distribution by projecting it onto a parametric family of distributions, and integrate it into a control heuristic based on the fully observable system to obtain a scalable policy. We demonstrate the effectiveness of our approach on several partially observed systems, including queueing systems and chemical reaction networks.

Cite this Paper

BibTeX
@InProceedings{pmlr-v238-eich24a, title = {Approximate Control for Continuous-Time {POMDPs}}, author = {Eich, Yannick and Alt, Bastian and Koeppl, Heinz}, booktitle = {Proceedings of The 27th International Conference on Artificial Intelligence and Statistics}, pages = {3160--3168}, year = {2024}, editor = {Dasgupta, Sanjoy and Mandt, Stephan and Li, Yingzhen}, volume = {238}, series = {Proceedings of Machine Learning Research}, month = {02--04 May}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v238/eich24a/eich24a.pdf}, url = {https://proceedings.mlr.press/v238/eich24a.html}, abstract = {This work proposes a decision-making framework for partially observable systems in continuous time with discrete state and action spaces. As optimal decision-making becomes intractable for large state spaces we employ approximation methods for the filtering and the control problem that scale well with an increasing number of states. Specifically, we approximate the high-dimensional filtering distribution by projecting it onto a parametric family of distributions, and integrate it into a control heuristic based on the fully observable system to obtain a scalable policy. We demonstrate the effectiveness of our approach on several partially observed systems, including queueing systems and chemical reaction networks.} }
Endnote
%0 Conference Paper %T Approximate Control for Continuous-Time POMDPs %A Yannick Eich %A Bastian Alt %A Heinz Koeppl %B Proceedings of The 27th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2024 %E Sanjoy Dasgupta %E Stephan Mandt %E Yingzhen Li %F pmlr-v238-eich24a %I PMLR %P 3160--3168 %U https://proceedings.mlr.press/v238/eich24a.html %V 238 %X This work proposes a decision-making framework for partially observable systems in continuous time with discrete state and action spaces. As optimal decision-making becomes intractable for large state spaces we employ approximation methods for the filtering and the control problem that scale well with an increasing number of states. Specifically, we approximate the high-dimensional filtering distribution by projecting it onto a parametric family of distributions, and integrate it into a control heuristic based on the fully observable system to obtain a scalable policy. We demonstrate the effectiveness of our approach on several partially observed systems, including queueing systems and chemical reaction networks.
APA
Eich, Y., Alt, B. & Koeppl, H.. (2024). Approximate Control for Continuous-Time POMDPs. Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 238:3160-3168 Available from https://proceedings.mlr.press/v238/eich24a.html.

Related Material