Variational Control for Guidance in Diffusion Models

Kushagra Pandey, Farrin Marouf Sofian, Felix Draxler, Theofanis Karaletsos, Stephan Mandt
Proceedings of the 42nd International Conference on Machine Learning, PMLR 267:47755-47780, 2025.

Abstract

Diffusion models exhibit excellent sample quality, but existing guidance methods often require additional model training or are limited to specific tasks. We revisit guidance in diffusion models from the perspective of variational inference and control, introducing Diffusion Trajectory Matching (DTM) that enables guiding pretrained diffusion trajectories to satisfy a terminal cost. DTM unifies a broad class of guidance methods and enables novel instantiations. We introduce a new method within this framework that achieves state-of-the-art results on several linear, non-linear, and blind inverse problems without requiring additional model training or specificity to pixel or latent space diffusion models. Our code will be available at https://github.com/czi-ai/oc-guidance.

Cite this Paper

BibTeX
@InProceedings{pmlr-v267-pandey25a, title = {Variational Control for Guidance in Diffusion Models}, author = {Pandey, Kushagra and Sofian, Farrin Marouf and Draxler, Felix and Karaletsos, Theofanis and Mandt, Stephan}, booktitle = {Proceedings of the 42nd International Conference on Machine Learning}, pages = {47755--47780}, year = {2025}, editor = {Singh, Aarti and Fazel, Maryam and Hsu, Daniel and Lacoste-Julien, Simon and Berkenkamp, Felix and Maharaj, Tegan and Wagstaff, Kiri and Zhu, Jerry}, volume = {267}, series = {Proceedings of Machine Learning Research}, month = {13--19 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v267/main/assets/pandey25a/pandey25a.pdf}, url = {https://proceedings.mlr.press/v267/pandey25a.html}, abstract = {Diffusion models exhibit excellent sample quality, but existing guidance methods often require additional model training or are limited to specific tasks. We revisit guidance in diffusion models from the perspective of variational inference and control, introducing Diffusion Trajectory Matching (DTM) that enables guiding pretrained diffusion trajectories to satisfy a terminal cost. DTM unifies a broad class of guidance methods and enables novel instantiations. We introduce a new method within this framework that achieves state-of-the-art results on several linear, non-linear, and blind inverse problems without requiring additional model training or specificity to pixel or latent space diffusion models. Our code will be available at https://github.com/czi-ai/oc-guidance.} }
Endnote
%0 Conference Paper %T Variational Control for Guidance in Diffusion Models %A Kushagra Pandey %A Farrin Marouf Sofian %A Felix Draxler %A Theofanis Karaletsos %A Stephan Mandt %B Proceedings of the 42nd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2025 %E Aarti Singh %E Maryam Fazel %E Daniel Hsu %E Simon Lacoste-Julien %E Felix Berkenkamp %E Tegan Maharaj %E Kiri Wagstaff %E Jerry Zhu %F pmlr-v267-pandey25a %I PMLR %P 47755--47780 %U https://proceedings.mlr.press/v267/pandey25a.html %V 267 %X Diffusion models exhibit excellent sample quality, but existing guidance methods often require additional model training or are limited to specific tasks. We revisit guidance in diffusion models from the perspective of variational inference and control, introducing Diffusion Trajectory Matching (DTM) that enables guiding pretrained diffusion trajectories to satisfy a terminal cost. DTM unifies a broad class of guidance methods and enables novel instantiations. We introduce a new method within this framework that achieves state-of-the-art results on several linear, non-linear, and blind inverse problems without requiring additional model training or specificity to pixel or latent space diffusion models. Our code will be available at https://github.com/czi-ai/oc-guidance.
APA
Pandey, K., Sofian, F.M., Draxler, F., Karaletsos, T. & Mandt, S.. (2025). Variational Control for Guidance in Diffusion Models. Proceedings of the 42nd International Conference on Machine Learning, in Proceedings of Machine Learning Research 267:47755-47780 Available from https://proceedings.mlr.press/v267/pandey25a.html.

Related Material