Plug-and-Play Methods Provably Converge with Properly Trained Denoisers

Ernest Ryu, Jialin Liu, Sicheng Wang, Xiaohan Chen, Zhangyang Wang, Wotao Yin
Proceedings of the 36th International Conference on Machine Learning, PMLR 97:5546-5557, 2019.

Abstract

Plug-and-play (PnP) is a non-convex framework that integrates modern denoising priors, such as BM3D or deep learning-based denoisers, into ADMM or other proximal algorithms. An advantage of PnP is that one can use pre-trained denoisers when there is not sufficient data for end-to-end training. Although PnP has been recently studied extensively with great empirical success, theoretical analysis addressing even the most basic question of convergence has been insufficient. In this paper, we theoretically establish convergence of PnP-FBS and PnP-ADMM, without using diminishing stepsizes, under a certain Lipschitz condition on the denoisers. We then propose real spectral normalization, a technique for training deep learning-based denoisers to satisfy the proposed Lipschitz condition. Finally, we present experimental results validating the theory.

Cite this Paper

BibTeX
@InProceedings{pmlr-v97-ryu19a, title = {Plug-and-Play Methods Provably Converge with Properly Trained Denoisers}, author = {Ryu, Ernest and Liu, Jialin and Wang, Sicheng and Chen, Xiaohan and Wang, Zhangyang and Yin, Wotao}, booktitle = {Proceedings of the 36th International Conference on Machine Learning}, pages = {5546--5557}, year = {2019}, editor = {Chaudhuri, Kamalika and Salakhutdinov, Ruslan}, volume = {97}, series = {Proceedings of Machine Learning Research}, month = {09--15 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v97/ryu19a/ryu19a.pdf}, url = {https://proceedings.mlr.press/v97/ryu19a.html}, abstract = {Plug-and-play (PnP) is a non-convex framework that integrates modern denoising priors, such as BM3D or deep learning-based denoisers, into ADMM or other proximal algorithms. An advantage of PnP is that one can use pre-trained denoisers when there is not sufficient data for end-to-end training. Although PnP has been recently studied extensively with great empirical success, theoretical analysis addressing even the most basic question of convergence has been insufficient. In this paper, we theoretically establish convergence of PnP-FBS and PnP-ADMM, without using diminishing stepsizes, under a certain Lipschitz condition on the denoisers. We then propose real spectral normalization, a technique for training deep learning-based denoisers to satisfy the proposed Lipschitz condition. Finally, we present experimental results validating the theory.} }
Endnote
%0 Conference Paper %T Plug-and-Play Methods Provably Converge with Properly Trained Denoisers %A Ernest Ryu %A Jialin Liu %A Sicheng Wang %A Xiaohan Chen %A Zhangyang Wang %A Wotao Yin %B Proceedings of the 36th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2019 %E Kamalika Chaudhuri %E Ruslan Salakhutdinov %F pmlr-v97-ryu19a %I PMLR %P 5546--5557 %U https://proceedings.mlr.press/v97/ryu19a.html %V 97 %X Plug-and-play (PnP) is a non-convex framework that integrates modern denoising priors, such as BM3D or deep learning-based denoisers, into ADMM or other proximal algorithms. An advantage of PnP is that one can use pre-trained denoisers when there is not sufficient data for end-to-end training. Although PnP has been recently studied extensively with great empirical success, theoretical analysis addressing even the most basic question of convergence has been insufficient. In this paper, we theoretically establish convergence of PnP-FBS and PnP-ADMM, without using diminishing stepsizes, under a certain Lipschitz condition on the denoisers. We then propose real spectral normalization, a technique for training deep learning-based denoisers to satisfy the proposed Lipschitz condition. Finally, we present experimental results validating the theory.
APA
Ryu, E., Liu, J., Wang, S., Chen, X., Wang, Z. & Yin, W.. (2019). Plug-and-Play Methods Provably Converge with Properly Trained Denoisers. Proceedings of the 36th International Conference on Machine Learning, in Proceedings of Machine Learning Research 97:5546-5557 Available from https://proceedings.mlr.press/v97/ryu19a.html.

Related Material