Particle Denoising Diffusion Sampler

Angus Phillips, Hai-Dang Dau, Michael John Hutchinson, Valentin De Bortoli, George Deligiannidis, Arnaud Doucet
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:40688-40724, 2024.

Abstract

Denoising diffusion models have become ubiquitous for generative modeling. The core idea is to transport the data distribution to a Gaussian by using a diffusion. Approximate samples from the data distribution are then obtained by estimating the time-reversal of this diffusion using score matching ideas. We follow here a similar strategy to sample from unnormalized probability densities and compute their normalizing constants. However, the time-reversed diffusion is here simulated by using an original iterative particle scheme relying on a novel score matching loss. Contrary to standard denoising diffusion models, the resulting Particle Denoising Diffusion Sampler (PDDS) provides asymptotically consistent estimates under mild assumptions. We demonstrate PDDS on multimodal and high dimensional sampling tasks.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-phillips24a, title = {Particle Denoising Diffusion Sampler}, author = {Phillips, Angus and Dau, Hai-Dang and Hutchinson, Michael John and De Bortoli, Valentin and Deligiannidis, George and Doucet, Arnaud}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {40688--40724}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/phillips24a/phillips24a.pdf}, url = {https://proceedings.mlr.press/v235/phillips24a.html}, abstract = {Denoising diffusion models have become ubiquitous for generative modeling. The core idea is to transport the data distribution to a Gaussian by using a diffusion. Approximate samples from the data distribution are then obtained by estimating the time-reversal of this diffusion using score matching ideas. We follow here a similar strategy to sample from unnormalized probability densities and compute their normalizing constants. However, the time-reversed diffusion is here simulated by using an original iterative particle scheme relying on a novel score matching loss. Contrary to standard denoising diffusion models, the resulting Particle Denoising Diffusion Sampler (PDDS) provides asymptotically consistent estimates under mild assumptions. We demonstrate PDDS on multimodal and high dimensional sampling tasks.} }
Endnote
%0 Conference Paper %T Particle Denoising Diffusion Sampler %A Angus Phillips %A Hai-Dang Dau %A Michael John Hutchinson %A Valentin De Bortoli %A George Deligiannidis %A Arnaud Doucet %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-phillips24a %I PMLR %P 40688--40724 %U https://proceedings.mlr.press/v235/phillips24a.html %V 235 %X Denoising diffusion models have become ubiquitous for generative modeling. The core idea is to transport the data distribution to a Gaussian by using a diffusion. Approximate samples from the data distribution are then obtained by estimating the time-reversal of this diffusion using score matching ideas. We follow here a similar strategy to sample from unnormalized probability densities and compute their normalizing constants. However, the time-reversed diffusion is here simulated by using an original iterative particle scheme relying on a novel score matching loss. Contrary to standard denoising diffusion models, the resulting Particle Denoising Diffusion Sampler (PDDS) provides asymptotically consistent estimates under mild assumptions. We demonstrate PDDS on multimodal and high dimensional sampling tasks.
APA
Phillips, A., Dau, H., Hutchinson, M.J., De Bortoli, V., Deligiannidis, G. & Doucet, A.. (2024). Particle Denoising Diffusion Sampler. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:40688-40724 Available from https://proceedings.mlr.press/v235/phillips24a.html.

Related Material