Conditional Kernel Imitation Learning for Continuous State Environments

Rishabh Agrawal, Nathan Dahlin, Rahul Jain, Ashutosh Nayyar
Proceedings of the 7th Annual Learning for Dynamics \& Control Conference, PMLR 283:469-483, 2025.

Abstract

Imitation Learning (IL) is an important paradigm within the broader reinforcement learning (RL) methodology. Unlike most of RL, it does not assume availability of reward-feedback. Classical methods such as behavioral cloning and inverse reinforcement learning are highly sensitive to estimation errors, especially in continuous state space problems. Meanwhile, state-of-the-art (SOTA) IL algorithms often require additional online interaction data to be effective. In this paper, we consider the problem of imitation learning in continuous state space environments based solely on observed behavior, without access to transition dynamics information, reward structure, or, most importantly, any additional interactions with the environment. Our approach is based on the Markov balance equation and introduces a novel conditional kernel density estimation-based imitation learning framework. It uses conditional kernel density estimators for transition dynamics and seeks to satisfy a balance equation for the environment. We establish that our estimators satisfy asymptotic consistency and present associated sample complexity analysis. Through a series of numerical experiments on continuous state benchmark environments, we show consistently superior empirical performance over many SOTA IL algorithms. The full paper with the appendix is available at: https://github.com/rishabh-1086/CKIL.

Cite this Paper

BibTeX
@InProceedings{pmlr-v283-agrawal25a, title = {Conditional Kernel Imitation Learning for Continuous State Environments}, author = {Agrawal, Rishabh and Dahlin, Nathan and Jain, Rahul and Nayyar, Ashutosh}, booktitle = {Proceedings of the 7th Annual Learning for Dynamics \& Control Conference}, pages = {469--483}, year = {2025}, editor = {Ozay, Necmiye and Balzano, Laura and Panagou, Dimitra and Abate, Alessandro}, volume = {283}, series = {Proceedings of Machine Learning Research}, month = {04--06 Jun}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v283/main/assets/agrawal25a/agrawal25a.pdf}, url = {https://proceedings.mlr.press/v283/agrawal25a.html}, abstract = {Imitation Learning (IL) is an important paradigm within the broader reinforcement learning (RL) methodology. Unlike most of RL, it does not assume availability of reward-feedback. Classical methods such as behavioral cloning and inverse reinforcement learning are highly sensitive to estimation errors, especially in continuous state space problems. Meanwhile, state-of-the-art (SOTA) IL algorithms often require additional online interaction data to be effective. In this paper, we consider the problem of imitation learning in continuous state space environments based solely on observed behavior, without access to transition dynamics information, reward structure, or, most importantly, any additional interactions with the environment. Our approach is based on the Markov balance equation and introduces a novel conditional kernel density estimation-based imitation learning framework. It uses conditional kernel density estimators for transition dynamics and seeks to satisfy a balance equation for the environment. We establish that our estimators satisfy asymptotic consistency and present associated sample complexity analysis. Through a series of numerical experiments on continuous state benchmark environments, we show consistently superior empirical performance over many SOTA IL algorithms. The full paper with the appendix is available at: https://github.com/rishabh-1086/CKIL.} }
Endnote
%0 Conference Paper %T Conditional Kernel Imitation Learning for Continuous State Environments %A Rishabh Agrawal %A Nathan Dahlin %A Rahul Jain %A Ashutosh Nayyar %B Proceedings of the 7th Annual Learning for Dynamics \& Control Conference %C Proceedings of Machine Learning Research %D 2025 %E Necmiye Ozay %E Laura Balzano %E Dimitra Panagou %E Alessandro Abate %F pmlr-v283-agrawal25a %I PMLR %P 469--483 %U https://proceedings.mlr.press/v283/agrawal25a.html %V 283 %X Imitation Learning (IL) is an important paradigm within the broader reinforcement learning (RL) methodology. Unlike most of RL, it does not assume availability of reward-feedback. Classical methods such as behavioral cloning and inverse reinforcement learning are highly sensitive to estimation errors, especially in continuous state space problems. Meanwhile, state-of-the-art (SOTA) IL algorithms often require additional online interaction data to be effective. In this paper, we consider the problem of imitation learning in continuous state space environments based solely on observed behavior, without access to transition dynamics information, reward structure, or, most importantly, any additional interactions with the environment. Our approach is based on the Markov balance equation and introduces a novel conditional kernel density estimation-based imitation learning framework. It uses conditional kernel density estimators for transition dynamics and seeks to satisfy a balance equation for the environment. We establish that our estimators satisfy asymptotic consistency and present associated sample complexity analysis. Through a series of numerical experiments on continuous state benchmark environments, we show consistently superior empirical performance over many SOTA IL algorithms. The full paper with the appendix is available at: https://github.com/rishabh-1086/CKIL.
APA
Agrawal, R., Dahlin, N., Jain, R. & Nayyar, A.. (2025). Conditional Kernel Imitation Learning for Continuous State Environments. Proceedings of the 7th Annual Learning for Dynamics \& Control Conference, in Proceedings of Machine Learning Research 283:469-483 Available from https://proceedings.mlr.press/v283/agrawal25a.html.

Related Material