Solving General Noisy Inverse Problem via Posterior Sampling: A Policy Gradient Viewpoint

Haoyue Tang, Tian Xie, Aosong Feng, Hanyu Wang, Chenyang Zhang, Yang Bai
Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, PMLR 238:2116-2124, 2024.

Abstract

Solving image inverse problems (e.g., super-resolution and inpainting) requires generating a high fidelity image that matches the given input (the low-resolution image or the masked image). By using the input image as guidance, we can leverage a pretrained diffusion generative model to solve a wide range of image inverse tasks without task specific model fine-tuning. To precisely estimate the guidance score function of the input image, we propose Diffusion Policy Gradient (DPG), a tractable computation method by viewing the intermediate noisy images as policies and the target image as the states selected by the policy. Experiments show that our method is robust to both Gaussian and Poisson noise degradation on multiple linear and non-linear inverse tasks, resulting into a higher image restoration quality on FFHQ, ImageNet and LSUN datasets.

Cite this Paper

BibTeX
@InProceedings{pmlr-v238-tang24b, title = {Solving General Noisy Inverse Problem via Posterior Sampling: A Policy Gradient Viewpoint}, author = {Tang, Haoyue and Xie, Tian and Feng, Aosong and Wang, Hanyu and Zhang, Chenyang and Bai, Yang}, booktitle = {Proceedings of The 27th International Conference on Artificial Intelligence and Statistics}, pages = {2116--2124}, year = {2024}, editor = {Dasgupta, Sanjoy and Mandt, Stephan and Li, Yingzhen}, volume = {238}, series = {Proceedings of Machine Learning Research}, month = {02--04 May}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v238/tang24b/tang24b.pdf}, url = {https://proceedings.mlr.press/v238/tang24b.html}, abstract = {Solving image inverse problems (e.g., super-resolution and inpainting) requires generating a high fidelity image that matches the given input (the low-resolution image or the masked image). By using the input image as guidance, we can leverage a pretrained diffusion generative model to solve a wide range of image inverse tasks without task specific model fine-tuning. To precisely estimate the guidance score function of the input image, we propose Diffusion Policy Gradient (DPG), a tractable computation method by viewing the intermediate noisy images as policies and the target image as the states selected by the policy. Experiments show that our method is robust to both Gaussian and Poisson noise degradation on multiple linear and non-linear inverse tasks, resulting into a higher image restoration quality on FFHQ, ImageNet and LSUN datasets.} }
Endnote
%0 Conference Paper %T Solving General Noisy Inverse Problem via Posterior Sampling: A Policy Gradient Viewpoint %A Haoyue Tang %A Tian Xie %A Aosong Feng %A Hanyu Wang %A Chenyang Zhang %A Yang Bai %B Proceedings of The 27th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2024 %E Sanjoy Dasgupta %E Stephan Mandt %E Yingzhen Li %F pmlr-v238-tang24b %I PMLR %P 2116--2124 %U https://proceedings.mlr.press/v238/tang24b.html %V 238 %X Solving image inverse problems (e.g., super-resolution and inpainting) requires generating a high fidelity image that matches the given input (the low-resolution image or the masked image). By using the input image as guidance, we can leverage a pretrained diffusion generative model to solve a wide range of image inverse tasks without task specific model fine-tuning. To precisely estimate the guidance score function of the input image, we propose Diffusion Policy Gradient (DPG), a tractable computation method by viewing the intermediate noisy images as policies and the target image as the states selected by the policy. Experiments show that our method is robust to both Gaussian and Poisson noise degradation on multiple linear and non-linear inverse tasks, resulting into a higher image restoration quality on FFHQ, ImageNet and LSUN datasets.
APA
Tang, H., Xie, T., Feng, A., Wang, H., Zhang, C. & Bai, Y.. (2024). Solving General Noisy Inverse Problem via Posterior Sampling: A Policy Gradient Viewpoint. Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 238:2116-2124 Available from https://proceedings.mlr.press/v238/tang24b.html.

Related Material