Latent Kullback Leibler Control for Continuous-State Systems using Probabilistic Graphical Models

Takamitsu Matsubara NAIST, Vicenç Gómez Radboud University Nijmegen, Hilbert Kappen Radboud University
Proceedings of the 30th Conference on Uncertainty in Artificial Intelligence, PMLR R12:371-380, 2014.

Abstract

Kullback Leibler (KL) control problems allow for efficient computation of optimal control by solving a principal eigenvector problem. However, direct applicability of such framework to continuous state-action systems is limited. In this paper, we propose to embed a KL control problem in a probabilistic graphical model where observed variables correspond to the continuous (possibly high-dimensional) state of the system and latent variables correspond to a discrete (low-dimensional) representation of the state amenable for KL control computation. We present two examples of this approach. The first one uses standard hidden Markov models (HMMs) and computes exact optimal control, but is only applicable to low-dimensional systems. The second one uses factorial HMMs, it is scalable to higher dimensional problems, but control computation is approximate. We illustrate both examples in several robot motor control tasks.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR12-naist14a, title = {Latent {K}ullback {L}eibler Control for Continuous-State Systems using Probabilistic Graphical Models}, author = {NAIST, Takamitsu Matsubara and Nijmegen, Vicen{\c{c}} G{\'o}mez Radboud University and University, Hilbert Kappen Radboud}, booktitle = {Proceedings of the 30th Conference on Uncertainty in Artificial Intelligence}, pages = {371--380}, year = {2014}, editor = {Zhang, Nevin L. and Tian, Jin}, volume = {R12}, series = {Proceedings of Machine Learning Research}, month = {23--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r12/main/assets/naist14a/naist14a.pdf}, url = {https://proceedings.mlr.press/r12/naist14a.html}, abstract = {Kullback Leibler (KL) control problems allow for efficient computation of optimal control by solving a principal eigenvector problem. However, direct applicability of such framework to continuous state-action systems is limited. In this paper, we propose to embed a KL control problem in a probabilistic graphical model where observed variables correspond to the continuous (possibly high-dimensional) state of the system and latent variables correspond to a discrete (low-dimensional) representation of the state amenable for KL control computation. We present two examples of this approach. The first one uses standard hidden Markov models (HMMs) and computes exact optimal control, but is only applicable to low-dimensional systems. The second one uses factorial HMMs, it is scalable to higher dimensional problems, but control computation is approximate. We illustrate both examples in several robot motor control tasks.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T Latent Kullback Leibler Control for Continuous-State Systems using Probabilistic Graphical Models %A Takamitsu Matsubara NAIST %A Vicenç Gómez Radboud University Nijmegen %A Hilbert Kappen Radboud University %B Proceedings of the 30th Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2014 %E Nevin L. Zhang %E Jin Tian %F pmlr-vR12-naist14a %I PMLR %P 371--380 %U https://proceedings.mlr.press/r12/naist14a.html %V R12 %X Kullback Leibler (KL) control problems allow for efficient computation of optimal control by solving a principal eigenvector problem. However, direct applicability of such framework to continuous state-action systems is limited. In this paper, we propose to embed a KL control problem in a probabilistic graphical model where observed variables correspond to the continuous (possibly high-dimensional) state of the system and latent variables correspond to a discrete (low-dimensional) representation of the state amenable for KL control computation. We present two examples of this approach. The first one uses standard hidden Markov models (HMMs) and computes exact optimal control, but is only applicable to low-dimensional systems. The second one uses factorial HMMs, it is scalable to higher dimensional problems, but control computation is approximate. We illustrate both examples in several robot motor control tasks. %Z Reissued by PMLR on 04 October 2026.
APA
NAIST, T.M., Nijmegen, V.G.R.U. & University, H.K.R.. (2014). Latent Kullback Leibler Control for Continuous-State Systems using Probabilistic Graphical Models. Proceedings of the 30th Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R12:371-380 Available from https://proceedings.mlr.press/r12/naist14a.html. Reissued by PMLR on 04 October 2026.

Related Material