Hilbert Space Embeddings of POMDPs

Yu Nishiyama, Abdeslam Boularias, Arthur Gretton, Kenji Fukumizu
Proceedings of the 28th Conference on Uncertainty in Artificial Intelligence, PMLR R10:643-652, 2012.

Abstract

A nonparametric approach for policy learning for POMDPs is proposed. The approach represents distributions over the states, observations, and actions as embeddings in feature spaces, which are reproducing kernel Hilbert spaces. Distributions over states given the observations are obtained by applying the kernel Bayes’ rule to these distribution embeddings. Policies and value functions are defined on the feature space over states, which leads to a feature space expression for the Bellman equation. Value iteration may then be used to estimate the optimal value function and associated policy. Experimental results confirm that the correct policy is learned using the feature space representation.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR10-nishiyama12a, title = {{H}ilbert Space Embeddings of POMDPs}, author = {Nishiyama, Yu and Boularias, Abdeslam and Gretton, Arthur and Fukumizu, Kenji}, booktitle = {Proceedings of the 28th Conference on Uncertainty in Artificial Intelligence}, pages = {643--652}, year = {2012}, editor = {de Freitas, Nando and Murphy, Kevin}, volume = {R10}, series = {Proceedings of Machine Learning Research}, month = {14--18 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r10/main/assets/nishiyama12a/nishiyama12a.pdf}, url = {https://proceedings.mlr.press/r10/nishiyama12a.html}, abstract = {A nonparametric approach for policy learning for POMDPs is proposed. The approach represents distributions over the states, observations, and actions as embeddings in feature spaces, which are reproducing kernel Hilbert spaces. Distributions over states given the observations are obtained by applying the kernel Bayes’ rule to these distribution embeddings. Policies and value functions are defined on the feature space over states, which leads to a feature space expression for the Bellman equation. Value iteration may then be used to estimate the optimal value function and associated policy. Experimental results confirm that the correct policy is learned using the feature space representation.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T Hilbert Space Embeddings of POMDPs %A Yu Nishiyama %A Abdeslam Boularias %A Arthur Gretton %A Kenji Fukumizu %B Proceedings of the 28th Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2012 %E Nando de Freitas %E Kevin Murphy %F pmlr-vR10-nishiyama12a %I PMLR %P 643--652 %U https://proceedings.mlr.press/r10/nishiyama12a.html %V R10 %X A nonparametric approach for policy learning for POMDPs is proposed. The approach represents distributions over the states, observations, and actions as embeddings in feature spaces, which are reproducing kernel Hilbert spaces. Distributions over states given the observations are obtained by applying the kernel Bayes’ rule to these distribution embeddings. Policies and value functions are defined on the feature space over states, which leads to a feature space expression for the Bellman equation. Value iteration may then be used to estimate the optimal value function and associated policy. Experimental results confirm that the correct policy is learned using the feature space representation. %Z Reissued by PMLR on 04 October 2026.
APA
Nishiyama, Y., Boularias, A., Gretton, A. & Fukumizu, K.. (2012). Hilbert Space Embeddings of POMDPs. Proceedings of the 28th Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R10:643-652 Available from https://proceedings.mlr.press/r10/nishiyama12a.html. Reissued by PMLR on 04 October 2026.

Related Material