SDDM: Score-Decomposed Diffusion Models on Manifolds for Unpaired Image-to-Image Translation

Shikun Sun, Longhui Wei, Junliang Xing, Jia Jia, Qi Tian
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:33115-33134, 2023.

Abstract

Recent score-based diffusion models (SBDMs) show promising results in unpaired image-to-image translation (I2I). However, existing methods, either energy-based or statistically-based, provide no explicit form of the interfered intermediate generative distributions. This work presents a new score-decomposed diffusion model (SDDM) on manifolds to explicitly optimize the tangled distributions during image generation. SDDM derives manifolds to make the distributions of adjacent time steps separable and decompose the score function or energy guidance into an image "denoising" part and a content "refinement" part. To refine the image in the same noise level, we equalize the refinement parts of the score function and energy guidance, which permits multi-objective optimization on the manifold. We also leverage the block adaptive instance normalization module to construct manifolds with lower dimensions but still concentrated with the perturbed reference image. SDDM outperforms existing SBDM-based methods with much fewer diffusion steps on several I2I benchmarks.

Cite this Paper

BibTeX
@InProceedings{pmlr-v202-sun23n, title = {{SDDM}: Score-Decomposed Diffusion Models on Manifolds for Unpaired Image-to-Image Translation}, author = {Sun, Shikun and Wei, Longhui and Xing, Junliang and Jia, Jia and Tian, Qi}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {33115--33134}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/sun23n/sun23n.pdf}, url = {https://proceedings.mlr.press/v202/sun23n.html}, abstract = {Recent score-based diffusion models (SBDMs) show promising results in unpaired image-to-image translation (I2I). However, existing methods, either energy-based or statistically-based, provide no explicit form of the interfered intermediate generative distributions. This work presents a new score-decomposed diffusion model (SDDM) on manifolds to explicitly optimize the tangled distributions during image generation. SDDM derives manifolds to make the distributions of adjacent time steps separable and decompose the score function or energy guidance into an image "denoising" part and a content "refinement" part. To refine the image in the same noise level, we equalize the refinement parts of the score function and energy guidance, which permits multi-objective optimization on the manifold. We also leverage the block adaptive instance normalization module to construct manifolds with lower dimensions but still concentrated with the perturbed reference image. SDDM outperforms existing SBDM-based methods with much fewer diffusion steps on several I2I benchmarks.} }
Endnote
%0 Conference Paper %T SDDM: Score-Decomposed Diffusion Models on Manifolds for Unpaired Image-to-Image Translation %A Shikun Sun %A Longhui Wei %A Junliang Xing %A Jia Jia %A Qi Tian %B Proceedings of the 40th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Andreas Krause %E Emma Brunskill %E Kyunghyun Cho %E Barbara Engelhardt %E Sivan Sabato %E Jonathan Scarlett %F pmlr-v202-sun23n %I PMLR %P 33115--33134 %U https://proceedings.mlr.press/v202/sun23n.html %V 202 %X Recent score-based diffusion models (SBDMs) show promising results in unpaired image-to-image translation (I2I). However, existing methods, either energy-based or statistically-based, provide no explicit form of the interfered intermediate generative distributions. This work presents a new score-decomposed diffusion model (SDDM) on manifolds to explicitly optimize the tangled distributions during image generation. SDDM derives manifolds to make the distributions of adjacent time steps separable and decompose the score function or energy guidance into an image "denoising" part and a content "refinement" part. To refine the image in the same noise level, we equalize the refinement parts of the score function and energy guidance, which permits multi-objective optimization on the manifold. We also leverage the block adaptive instance normalization module to construct manifolds with lower dimensions but still concentrated with the perturbed reference image. SDDM outperforms existing SBDM-based methods with much fewer diffusion steps on several I2I benchmarks.
APA
Sun, S., Wei, L., Xing, J., Jia, J. & Tian, Q.. (2023). SDDM: Score-Decomposed Diffusion Models on Manifolds for Unpaired Image-to-Image Translation. Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:33115-33134 Available from https://proceedings.mlr.press/v202/sun23n.html.

Related Material