User-defined Event Sampling and Uncertainty Quantification in Diffusion Models for Physical Dynamical Systems

Marc Anton Finzi, Anudhyan Boral, Andrew Gordon Wilson, Fei Sha, Leonardo Zepeda-Nunez
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:10136-10152, 2023.

Abstract

Diffusion models are a class of probabilistic generative models that have been widely used as a prior for image processing tasks like text conditional generation and inpainting. We demonstrate that these models can be adapted to make predictions and provide uncertainty quantification for chaotic dynamical systems. In these applications, diffusion models can implicitly represent knowledge about outliers and extreme events; however, querying that knowledge through conditional sampling or measuring probabilities is surprisingly difficult. Existing methods for conditional sampling at inference time seek mainly to enforce the constraints, which is insufficient to match the statistics of the distribution or compute the probability of the chosen events. To achieve these ends, optimally one would use the conditional score function, but its computation is typically intractable. In this work, we develop a probabilistic approximation scheme for the conditional score function which provably converges to the true distribution as the noise level decreases. With this scheme we are able to sample conditionally on nonlinear user-defined events at inference time, and matches data statistics even when sampling from the tails of the distribution.

Cite this Paper

BibTeX
@InProceedings{pmlr-v202-finzi23a, title = {User-defined Event Sampling and Uncertainty Quantification in Diffusion Models for Physical Dynamical Systems}, author = {Finzi, Marc Anton and Boral, Anudhyan and Wilson, Andrew Gordon and Sha, Fei and Zepeda-Nunez, Leonardo}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {10136--10152}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/finzi23a/finzi23a.pdf}, url = {https://proceedings.mlr.press/v202/finzi23a.html}, abstract = {Diffusion models are a class of probabilistic generative models that have been widely used as a prior for image processing tasks like text conditional generation and inpainting. We demonstrate that these models can be adapted to make predictions and provide uncertainty quantification for chaotic dynamical systems. In these applications, diffusion models can implicitly represent knowledge about outliers and extreme events; however, querying that knowledge through conditional sampling or measuring probabilities is surprisingly difficult. Existing methods for conditional sampling at inference time seek mainly to enforce the constraints, which is insufficient to match the statistics of the distribution or compute the probability of the chosen events. To achieve these ends, optimally one would use the conditional score function, but its computation is typically intractable. In this work, we develop a probabilistic approximation scheme for the conditional score function which provably converges to the true distribution as the noise level decreases. With this scheme we are able to sample conditionally on nonlinear user-defined events at inference time, and matches data statistics even when sampling from the tails of the distribution.} }
Endnote
%0 Conference Paper %T User-defined Event Sampling and Uncertainty Quantification in Diffusion Models for Physical Dynamical Systems %A Marc Anton Finzi %A Anudhyan Boral %A Andrew Gordon Wilson %A Fei Sha %A Leonardo Zepeda-Nunez %B Proceedings of the 40th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Andreas Krause %E Emma Brunskill %E Kyunghyun Cho %E Barbara Engelhardt %E Sivan Sabato %E Jonathan Scarlett %F pmlr-v202-finzi23a %I PMLR %P 10136--10152 %U https://proceedings.mlr.press/v202/finzi23a.html %V 202 %X Diffusion models are a class of probabilistic generative models that have been widely used as a prior for image processing tasks like text conditional generation and inpainting. We demonstrate that these models can be adapted to make predictions and provide uncertainty quantification for chaotic dynamical systems. In these applications, diffusion models can implicitly represent knowledge about outliers and extreme events; however, querying that knowledge through conditional sampling or measuring probabilities is surprisingly difficult. Existing methods for conditional sampling at inference time seek mainly to enforce the constraints, which is insufficient to match the statistics of the distribution or compute the probability of the chosen events. To achieve these ends, optimally one would use the conditional score function, but its computation is typically intractable. In this work, we develop a probabilistic approximation scheme for the conditional score function which provably converges to the true distribution as the noise level decreases. With this scheme we are able to sample conditionally on nonlinear user-defined events at inference time, and matches data statistics even when sampling from the tails of the distribution.
APA
Finzi, M.A., Boral, A., Wilson, A.G., Sha, F. & Zepeda-Nunez, L.. (2023). User-defined Event Sampling and Uncertainty Quantification in Diffusion Models for Physical Dynamical Systems. Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:10136-10152 Available from https://proceedings.mlr.press/v202/finzi23a.html.

Related Material