DualCOIL: Offline Imitation Learning from Contrasting Demonstrations

Huy Hoang, Tien Anh Mai, Pradeep Varakantham, Tanvi Verma
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:43300-43335, 2026.

Abstract

Offline imitation learning typically learns from expert and unlabeled demonstrations, yet often overlooks the valuable signal in explicitly undesirable behaviors. In this work, we study offline imitation learning from contrasting behaviors, where the dataset contains both expert and undesirable demonstrations along with an unlabeled set of demonstrations. We propose a novel formulation that optimizes a difference of KL divergences over the state-action visitation distributions of expert and undesirable (or bad) data. Although the resulting objective is a DC (Difference-of-Convex) program, we prove that it becomes convex when expert demonstrations outweigh undesirable demonstrations, enabling a practical and stable non-adversarial training objective. Our method avoids adversarial training and handles both positive and negative demonstrations in a unified framework. Extensive experiments on standard offline imitation learning benchmarks demonstrate that our approach consistently outperforms state-of-the-art baselines.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-hoang26c, title = {{D}ual{COIL}: Offline Imitation Learning from Contrasting Demonstrations}, author = {Hoang, Huy and Mai, Tien Anh and Varakantham, Pradeep and Verma, Tanvi}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {43300--43335}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/hoang26c/hoang26c.pdf}, url = {https://proceedings.mlr.press/v306/hoang26c.html}, abstract = {Offline imitation learning typically learns from expert and unlabeled demonstrations, yet often overlooks the valuable signal in explicitly undesirable behaviors. In this work, we study offline imitation learning from contrasting behaviors, where the dataset contains both expert and undesirable demonstrations along with an unlabeled set of demonstrations. We propose a novel formulation that optimizes a difference of KL divergences over the state-action visitation distributions of expert and undesirable (or bad) data. Although the resulting objective is a DC (Difference-of-Convex) program, we prove that it becomes convex when expert demonstrations outweigh undesirable demonstrations, enabling a practical and stable non-adversarial training objective. Our method avoids adversarial training and handles both positive and negative demonstrations in a unified framework. Extensive experiments on standard offline imitation learning benchmarks demonstrate that our approach consistently outperforms state-of-the-art baselines.} }
Endnote
%0 Conference Paper %T DualCOIL: Offline Imitation Learning from Contrasting Demonstrations %A Huy Hoang %A Tien Anh Mai %A Pradeep Varakantham %A Tanvi Verma %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-hoang26c %I PMLR %P 43300--43335 %U https://proceedings.mlr.press/v306/hoang26c.html %V 306 %X Offline imitation learning typically learns from expert and unlabeled demonstrations, yet often overlooks the valuable signal in explicitly undesirable behaviors. In this work, we study offline imitation learning from contrasting behaviors, where the dataset contains both expert and undesirable demonstrations along with an unlabeled set of demonstrations. We propose a novel formulation that optimizes a difference of KL divergences over the state-action visitation distributions of expert and undesirable (or bad) data. Although the resulting objective is a DC (Difference-of-Convex) program, we prove that it becomes convex when expert demonstrations outweigh undesirable demonstrations, enabling a practical and stable non-adversarial training objective. Our method avoids adversarial training and handles both positive and negative demonstrations in a unified framework. Extensive experiments on standard offline imitation learning benchmarks demonstrate that our approach consistently outperforms state-of-the-art baselines.
APA
Hoang, H., Mai, T.A., Varakantham, P. & Verma, T.. (2026). DualCOIL: Offline Imitation Learning from Contrasting Demonstrations. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:43300-43335 Available from https://proceedings.mlr.press/v306/hoang26c.html.

Related Material