Inverse Reinforcement Learning via Deep Gaussian Process

Ming Jin, Andreas Damianou, Pieter Abbeel, Costas Spanos
Proceedings of the 33rd Conference on Uncertainty in Artificial Intelligence, PMLR R15:471-480, 2017.

Abstract

We propose a new approach to inverse reinforcement learning (IRL) based on the deep Gaussian process (deep GP) model, which is capable of learning complicated reward structures with few demonstrations. Our model stacks multiple latent GP layers to learn abstract representations of the state feature space, which is linked to the demonstrations through the Maximum Entropy learning framework. Incorporating the IRL engine into the nonlinear latent structure renders existing deep GP inference approaches intractable. To tackle this, we develop a non-standard variational approximation framework which extends previous inference schemes. This allows for approximate Bayesian treatment of the feature space and guards against overfitting. Carrying out representation and inverse reinforcement learning simultaneously within our model outperforms state-of-the-art approaches, as we demonstrate with experiments on standard benchmarks (“object world”,“highway driving”) and a new benchmark (“binary world”).

Cite this Paper

BibTeX
@InProceedings{pmlr-vR15-jin17a, title = {Inverse Reinforcement Learning via Deep {G}aussian Process}, author = {Jin, Ming and Damianou, Andreas and Abbeel, Pieter and Spanos, Costas}, booktitle = {Proceedings of the 33rd Conference on Uncertainty in Artificial Intelligence}, pages = {471--480}, year = {2017}, editor = {Elidan, Gal and Kersting, Kristian}, volume = {R15}, series = {Proceedings of Machine Learning Research}, month = {11--15 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r15/main/assets/jin17a/jin17a.pdf}, url = {https://proceedings.mlr.press/r15/jin17a.html}, abstract = {We propose a new approach to inverse reinforcement learning (IRL) based on the deep Gaussian process (deep GP) model, which is capable of learning complicated reward structures with few demonstrations. Our model stacks multiple latent GP layers to learn abstract representations of the state feature space, which is linked to the demonstrations through the Maximum Entropy learning framework. Incorporating the IRL engine into the nonlinear latent structure renders existing deep GP inference approaches intractable. To tackle this, we develop a non-standard variational approximation framework which extends previous inference schemes. This allows for approximate Bayesian treatment of the feature space and guards against overfitting. Carrying out representation and inverse reinforcement learning simultaneously within our model outperforms state-of-the-art approaches, as we demonstrate with experiments on standard benchmarks (“object world”,“highway driving”) and a new benchmark (“binary world”).}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T Inverse Reinforcement Learning via Deep Gaussian Process %A Ming Jin %A Andreas Damianou %A Pieter Abbeel %A Costas Spanos %B Proceedings of the 33rd Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2017 %E Gal Elidan %E Kristian Kersting %F pmlr-vR15-jin17a %I PMLR %P 471--480 %U https://proceedings.mlr.press/r15/jin17a.html %V R15 %X We propose a new approach to inverse reinforcement learning (IRL) based on the deep Gaussian process (deep GP) model, which is capable of learning complicated reward structures with few demonstrations. Our model stacks multiple latent GP layers to learn abstract representations of the state feature space, which is linked to the demonstrations through the Maximum Entropy learning framework. Incorporating the IRL engine into the nonlinear latent structure renders existing deep GP inference approaches intractable. To tackle this, we develop a non-standard variational approximation framework which extends previous inference schemes. This allows for approximate Bayesian treatment of the feature space and guards against overfitting. Carrying out representation and inverse reinforcement learning simultaneously within our model outperforms state-of-the-art approaches, as we demonstrate with experiments on standard benchmarks (“object world”,“highway driving”) and a new benchmark (“binary world”). %Z Reissued by PMLR on 04 October 2026.
APA
Jin, M., Damianou, A., Abbeel, P. & Spanos, C.. (2017). Inverse Reinforcement Learning via Deep Gaussian Process. Proceedings of the 33rd Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R15:471-480 Available from https://proceedings.mlr.press/r15/jin17a.html. Reissued by PMLR on 04 October 2026.

Related Material