Hilbert Space Embeddings of Predictive State Representations

Byron Boots, Geoffrey Gordon, Arthur Gretton
Proceedings of the 29th Conference on Uncertainty in Artificial Intelligence, PMLR R11:45-54, 2013.

Abstract

Predictive State Representations (PSRs) are an expressive class of models for controlled stochastic processes. PSRs represent state as a set of predictions of future observable events. Because PSRs are defined entirely in terms of observable data, statistically con- sistent estimates of PSR parameters can be learned efficiently by manipulating moments of observed training data. Most learning al- gorithms for PSRs have assumed that actions and observations are finite with low cardinal- ity. In this paper, we generalize PSRs to in- finite sets of observations and actions, using the recent concept of Hilbert space embed- dings of distributions. The essence is to rep- resent the state as one or more nonparamet- ric conditional embedding operators in a Re- producing Kernel Hilbert Space (RKHS) and leverage recent work in kernel methods to es- timate, predict, and update the representa- tion. We show that these Hilbert space em- beddings of PSRs are able to gracefully han- dle continuous actions and observations, and that our learned models outperform compet- ing system identification algorithms on sev- eral prediction benchmarks.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR11-boots13a, title = {{H}ilbert Space Embeddings of Predictive State Representations}, author = {Boots, Byron and Gordon, Geoffrey and Gretton, Arthur}, booktitle = {Proceedings of the 29th Conference on Uncertainty in Artificial Intelligence}, pages = {45--54}, year = {2013}, editor = {Nicholson, Ann and Smyth, Padhraic}, volume = {R11}, series = {Proceedings of Machine Learning Research}, month = {12--14 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r11/main/assets/boots13a/boots13a.pdf}, url = {https://proceedings.mlr.press/r11/boots13a.html}, abstract = {Predictive State Representations (PSRs) are an expressive class of models for controlled stochastic processes. PSRs represent state as a set of predictions of future observable events. Because PSRs are defined entirely in terms of observable data, statistically con- sistent estimates of PSR parameters can be learned efficiently by manipulating moments of observed training data. Most learning al- gorithms for PSRs have assumed that actions and observations are finite with low cardinal- ity. In this paper, we generalize PSRs to in- finite sets of observations and actions, using the recent concept of Hilbert space embed- dings of distributions. The essence is to rep- resent the state as one or more nonparamet- ric conditional embedding operators in a Re- producing Kernel Hilbert Space (RKHS) and leverage recent work in kernel methods to es- timate, predict, and update the representa- tion. We show that these Hilbert space em- beddings of PSRs are able to gracefully han- dle continuous actions and observations, and that our learned models outperform compet- ing system identification algorithms on sev- eral prediction benchmarks.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T Hilbert Space Embeddings of Predictive State Representations %A Byron Boots %A Geoffrey Gordon %A Arthur Gretton %B Proceedings of the 29th Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2013 %E Ann Nicholson %E Padhraic Smyth %F pmlr-vR11-boots13a %I PMLR %P 45--54 %U https://proceedings.mlr.press/r11/boots13a.html %V R11 %X Predictive State Representations (PSRs) are an expressive class of models for controlled stochastic processes. PSRs represent state as a set of predictions of future observable events. Because PSRs are defined entirely in terms of observable data, statistically con- sistent estimates of PSR parameters can be learned efficiently by manipulating moments of observed training data. Most learning al- gorithms for PSRs have assumed that actions and observations are finite with low cardinal- ity. In this paper, we generalize PSRs to in- finite sets of observations and actions, using the recent concept of Hilbert space embed- dings of distributions. The essence is to rep- resent the state as one or more nonparamet- ric conditional embedding operators in a Re- producing Kernel Hilbert Space (RKHS) and leverage recent work in kernel methods to es- timate, predict, and update the representa- tion. We show that these Hilbert space em- beddings of PSRs are able to gracefully han- dle continuous actions and observations, and that our learned models outperform compet- ing system identification algorithms on sev- eral prediction benchmarks. %Z Reissued by PMLR on 04 October 2026.
APA
Boots, B., Gordon, G. & Gretton, A.. (2013). Hilbert Space Embeddings of Predictive State Representations. Proceedings of the 29th Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R11:45-54 Available from https://proceedings.mlr.press/r11/boots13a.html. Reissued by PMLR on 04 October 2026.

Related Material