Going beyond Compositions, DDPMs Can Produce Zero-Shot Interpolations

Justin Deschenaux, Igor Krawczuk, Grigorios Chrysos, Volkan Cevher
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:10597-10623, 2024.

Abstract

Denoising Diffusion Probabilistic Models (DDPMs) exhibit remarkable capabilities in image generation, with studies suggesting that they can generalize by composing latent factors learned from the training data. In this work, we go further and study DDPMs trained on strictly separate subsets of the data distribution with large gaps on the support of the latent factors. We show that such a model can effectively generate images in the unexplored, intermediate regions of the distribution. For instance, when trained on clearly smiling and non-smiling faces, we demonstrate a sampling procedure which can generate slightly smiling faces without reference images (zero-shot interpolation). We replicate these findings for other attributes as well as other datasets. Our code is available on GitHub.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-deschenaux24a, title = {Going beyond Compositions, {DDPM}s Can Produce Zero-Shot Interpolations}, author = {Deschenaux, Justin and Krawczuk, Igor and Chrysos, Grigorios and Cevher, Volkan}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {10597--10623}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/deschenaux24a/deschenaux24a.pdf}, url = {https://proceedings.mlr.press/v235/deschenaux24a.html}, abstract = {Denoising Diffusion Probabilistic Models (DDPMs) exhibit remarkable capabilities in image generation, with studies suggesting that they can generalize by composing latent factors learned from the training data. In this work, we go further and study DDPMs trained on strictly separate subsets of the data distribution with large gaps on the support of the latent factors. We show that such a model can effectively generate images in the unexplored, intermediate regions of the distribution. For instance, when trained on clearly smiling and non-smiling faces, we demonstrate a sampling procedure which can generate slightly smiling faces without reference images (zero-shot interpolation). We replicate these findings for other attributes as well as other datasets. Our code is available on GitHub.} }
Endnote
%0 Conference Paper %T Going beyond Compositions, DDPMs Can Produce Zero-Shot Interpolations %A Justin Deschenaux %A Igor Krawczuk %A Grigorios Chrysos %A Volkan Cevher %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-deschenaux24a %I PMLR %P 10597--10623 %U https://proceedings.mlr.press/v235/deschenaux24a.html %V 235 %X Denoising Diffusion Probabilistic Models (DDPMs) exhibit remarkable capabilities in image generation, with studies suggesting that they can generalize by composing latent factors learned from the training data. In this work, we go further and study DDPMs trained on strictly separate subsets of the data distribution with large gaps on the support of the latent factors. We show that such a model can effectively generate images in the unexplored, intermediate regions of the distribution. For instance, when trained on clearly smiling and non-smiling faces, we demonstrate a sampling procedure which can generate slightly smiling faces without reference images (zero-shot interpolation). We replicate these findings for other attributes as well as other datasets. Our code is available on GitHub.
APA
Deschenaux, J., Krawczuk, I., Chrysos, G. & Cevher, V.. (2024). Going beyond Compositions, DDPMs Can Produce Zero-Shot Interpolations. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:10597-10623 Available from https://proceedings.mlr.press/v235/deschenaux24a.html.

Related Material