Understanding Diffusion Models by Feynman’s Path Integral

Yuji Hirono, Akinori Tanaka, Kenji Fukushima
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:18324-18351, 2024.

Abstract

Score-based diffusion models have proven effective in image generation and have gained widespread usage; however, the underlying factors contributing to the performance disparity between stochastic and deterministic (i.e., the probability flow ODEs) sampling schemes remain unclear. We introduce a novel formulation of diffusion models using Feynman’s path integral, which is a formulation originally developed for quantum physics. We find this formulation providing comprehensive descriptions of score-based generative models, and demonstrate the derivation of backward stochastic differential equations and loss functions. The formulation accommodates an interpolating parameter connecting stochastic and deterministic sampling schemes, and we identify this parameter as a counterpart of Planck’s constant in quantum physics. This analogy enables us to apply the Wentzel–Kramers–Brillouin (WKB) expansion, a well-established technique in quantum physics, for evaluating the negative log-likelihood to assess the performance disparity between stochastic and deterministic sampling schemes.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-hirono24a, title = {Understanding Diffusion Models by Feynman’s Path Integral}, author = {Hirono, Yuji and Tanaka, Akinori and Fukushima, Kenji}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {18324--18351}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/hirono24a/hirono24a.pdf}, url = {https://proceedings.mlr.press/v235/hirono24a.html}, abstract = {Score-based diffusion models have proven effective in image generation and have gained widespread usage; however, the underlying factors contributing to the performance disparity between stochastic and deterministic (i.e., the probability flow ODEs) sampling schemes remain unclear. We introduce a novel formulation of diffusion models using Feynman’s path integral, which is a formulation originally developed for quantum physics. We find this formulation providing comprehensive descriptions of score-based generative models, and demonstrate the derivation of backward stochastic differential equations and loss functions. The formulation accommodates an interpolating parameter connecting stochastic and deterministic sampling schemes, and we identify this parameter as a counterpart of Planck’s constant in quantum physics. This analogy enables us to apply the Wentzel–Kramers–Brillouin (WKB) expansion, a well-established technique in quantum physics, for evaluating the negative log-likelihood to assess the performance disparity between stochastic and deterministic sampling schemes.} }
Endnote
%0 Conference Paper %T Understanding Diffusion Models by Feynman’s Path Integral %A Yuji Hirono %A Akinori Tanaka %A Kenji Fukushima %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-hirono24a %I PMLR %P 18324--18351 %U https://proceedings.mlr.press/v235/hirono24a.html %V 235 %X Score-based diffusion models have proven effective in image generation and have gained widespread usage; however, the underlying factors contributing to the performance disparity between stochastic and deterministic (i.e., the probability flow ODEs) sampling schemes remain unclear. We introduce a novel formulation of diffusion models using Feynman’s path integral, which is a formulation originally developed for quantum physics. We find this formulation providing comprehensive descriptions of score-based generative models, and demonstrate the derivation of backward stochastic differential equations and loss functions. The formulation accommodates an interpolating parameter connecting stochastic and deterministic sampling schemes, and we identify this parameter as a counterpart of Planck’s constant in quantum physics. This analogy enables us to apply the Wentzel–Kramers–Brillouin (WKB) expansion, a well-established technique in quantum physics, for evaluating the negative log-likelihood to assess the performance disparity between stochastic and deterministic sampling schemes.
APA
Hirono, Y., Tanaka, A. & Fukushima, K.. (2024). Understanding Diffusion Models by Feynman’s Path Integral. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:18324-18351 Available from https://proceedings.mlr.press/v235/hirono24a.html.

Related Material