Soft then Hard: Rethinking the Quantization in Neural Image Compression

Zongyu Guo, Zhizheng Zhang, Runsen Feng, Zhibo Chen
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:3920-3929, 2021.

Abstract

Quantization is one of the core components in lossy image compression. For neural image compression, end-to-end optimization requires differentiable approximations of quantization, which can generally be grouped into three categories: additive uniform noise, straight-through estimator and soft-to-hard annealing. Training with additive uniform noise approximates the quantization error variationally but suffers from the train-test mismatch. The other two methods do not encounter this mismatch but, as shown in this paper, hurt the rate-distortion performance since the latent representation ability is weakened. We thus propose a novel soft-then-hard quantization strategy for neural image compression that first learns an expressive latent space softly, then closes the train-test mismatch with hard quantization. In addition, beyond the fixed integer-quantization, we apply scaled additive uniform noise to adaptively control the quantization granularity by deriving a new variational upper bound on actual rate. Experiments demonstrate that our proposed methods are easy to adopt, stable to train, and highly effective especially on complex compression models.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-guo21c, title = {Soft then Hard: Rethinking the Quantization in Neural Image Compression}, author = {Guo, Zongyu and Zhang, Zhizheng and Feng, Runsen and Chen, Zhibo}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {3920--3929}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/guo21c/guo21c.pdf}, url = {https://proceedings.mlr.press/v139/guo21c.html}, abstract = {Quantization is one of the core components in lossy image compression. For neural image compression, end-to-end optimization requires differentiable approximations of quantization, which can generally be grouped into three categories: additive uniform noise, straight-through estimator and soft-to-hard annealing. Training with additive uniform noise approximates the quantization error variationally but suffers from the train-test mismatch. The other two methods do not encounter this mismatch but, as shown in this paper, hurt the rate-distortion performance since the latent representation ability is weakened. We thus propose a novel soft-then-hard quantization strategy for neural image compression that first learns an expressive latent space softly, then closes the train-test mismatch with hard quantization. In addition, beyond the fixed integer-quantization, we apply scaled additive uniform noise to adaptively control the quantization granularity by deriving a new variational upper bound on actual rate. Experiments demonstrate that our proposed methods are easy to adopt, stable to train, and highly effective especially on complex compression models.} }
Endnote
%0 Conference Paper %T Soft then Hard: Rethinking the Quantization in Neural Image Compression %A Zongyu Guo %A Zhizheng Zhang %A Runsen Feng %A Zhibo Chen %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-guo21c %I PMLR %P 3920--3929 %U https://proceedings.mlr.press/v139/guo21c.html %V 139 %X Quantization is one of the core components in lossy image compression. For neural image compression, end-to-end optimization requires differentiable approximations of quantization, which can generally be grouped into three categories: additive uniform noise, straight-through estimator and soft-to-hard annealing. Training with additive uniform noise approximates the quantization error variationally but suffers from the train-test mismatch. The other two methods do not encounter this mismatch but, as shown in this paper, hurt the rate-distortion performance since the latent representation ability is weakened. We thus propose a novel soft-then-hard quantization strategy for neural image compression that first learns an expressive latent space softly, then closes the train-test mismatch with hard quantization. In addition, beyond the fixed integer-quantization, we apply scaled additive uniform noise to adaptively control the quantization granularity by deriving a new variational upper bound on actual rate. Experiments demonstrate that our proposed methods are easy to adopt, stable to train, and highly effective especially on complex compression models.
APA
Guo, Z., Zhang, Z., Feng, R. & Chen, Z.. (2021). Soft then Hard: Rethinking the Quantization in Neural Image Compression. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:3920-3929 Available from https://proceedings.mlr.press/v139/guo21c.html.

Related Material