The Perception-Robustness Tradeoff in Deterministic Image Restoration

Guy Ohayon, Tomer Michaeli, Michael Elad
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:38599-38638, 2024.

Abstract

We study the behavior of deterministic methods for solving inverse problems in imaging. These methods are commonly designed to achieve two goals: (1) attaining high perceptual quality, and (2) generating reconstructions that are consistent with the measurements. We provide a rigorous proof that the better a predictor satisfies these two requirements, the larger its Lipschitz constant must be, regardless of the nature of the degradation involved. In particular, to approach perfect perceptual quality and perfect consistency, the Lipschitz constant of the model must grow to infinity. This implies that such methods are necessarily more susceptible to adversarial attacks. We demonstrate our theory on single image super-resolution algorithms, addressing both noisy and noiseless settings. We also show how this undesired behavior can be leveraged to explore the posterior distribution, thereby allowing the deterministic model to imitate stochastic methods.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-ohayon24a, title = {The Perception-Robustness Tradeoff in Deterministic Image Restoration}, author = {Ohayon, Guy and Michaeli, Tomer and Elad, Michael}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {38599--38638}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/ohayon24a/ohayon24a.pdf}, url = {https://proceedings.mlr.press/v235/ohayon24a.html}, abstract = {We study the behavior of deterministic methods for solving inverse problems in imaging. These methods are commonly designed to achieve two goals: (1) attaining high perceptual quality, and (2) generating reconstructions that are consistent with the measurements. We provide a rigorous proof that the better a predictor satisfies these two requirements, the larger its Lipschitz constant must be, regardless of the nature of the degradation involved. In particular, to approach perfect perceptual quality and perfect consistency, the Lipschitz constant of the model must grow to infinity. This implies that such methods are necessarily more susceptible to adversarial attacks. We demonstrate our theory on single image super-resolution algorithms, addressing both noisy and noiseless settings. We also show how this undesired behavior can be leveraged to explore the posterior distribution, thereby allowing the deterministic model to imitate stochastic methods.} }
Endnote
%0 Conference Paper %T The Perception-Robustness Tradeoff in Deterministic Image Restoration %A Guy Ohayon %A Tomer Michaeli %A Michael Elad %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-ohayon24a %I PMLR %P 38599--38638 %U https://proceedings.mlr.press/v235/ohayon24a.html %V 235 %X We study the behavior of deterministic methods for solving inverse problems in imaging. These methods are commonly designed to achieve two goals: (1) attaining high perceptual quality, and (2) generating reconstructions that are consistent with the measurements. We provide a rigorous proof that the better a predictor satisfies these two requirements, the larger its Lipschitz constant must be, regardless of the nature of the degradation involved. In particular, to approach perfect perceptual quality and perfect consistency, the Lipschitz constant of the model must grow to infinity. This implies that such methods are necessarily more susceptible to adversarial attacks. We demonstrate our theory on single image super-resolution algorithms, addressing both noisy and noiseless settings. We also show how this undesired behavior can be leveraged to explore the posterior distribution, thereby allowing the deterministic model to imitate stochastic methods.
APA
Ohayon, G., Michaeli, T. & Elad, M.. (2024). The Perception-Robustness Tradeoff in Deterministic Image Restoration. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:38599-38638 Available from https://proceedings.mlr.press/v235/ohayon24a.html.

Related Material