Importance Matching Lemma for Lossy Compression with Side Information

Buu Phan, Ashish Khisti, Christos Louizos
Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, PMLR 238:1387-1395, 2024.

Abstract

We propose two extensions to existing importance sampling based methods for lossy compression. First, we introduce an importance sampling based compression scheme that is a variant of ordered random coding (Theis and Ahmed, 2022) and is amenable to direct evaluation of the achievable compression rate for a finite number of samples. Our second and major contribution is the \emph{importance matching lemma}, which is a finite proposal counterpart of the recently introduced {Poisson matching lemma} (Li and Anantharam, 2021). By integrating with deep learning, we provide a new coding scheme for distributed lossy compression with side information at the decoder. We demonstrate the effectiveness of the proposed scheme through experiments involving synthetic Gaussian sources, distributed image compression with MNIST and vertical federated learning with CIFAR-10.

Cite this Paper

BibTeX
@InProceedings{pmlr-v238-phan24a, title = {Importance Matching Lemma for Lossy Compression with Side Information}, author = {Phan, Buu and Khisti, Ashish and Louizos, Christos}, booktitle = {Proceedings of The 27th International Conference on Artificial Intelligence and Statistics}, pages = {1387--1395}, year = {2024}, editor = {Dasgupta, Sanjoy and Mandt, Stephan and Li, Yingzhen}, volume = {238}, series = {Proceedings of Machine Learning Research}, month = {02--04 May}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v238/phan24a/phan24a.pdf}, url = {https://proceedings.mlr.press/v238/phan24a.html}, abstract = {We propose two extensions to existing importance sampling based methods for lossy compression. First, we introduce an importance sampling based compression scheme that is a variant of ordered random coding (Theis and Ahmed, 2022) and is amenable to direct evaluation of the achievable compression rate for a finite number of samples. Our second and major contribution is the \emph{importance matching lemma}, which is a finite proposal counterpart of the recently introduced {Poisson matching lemma} (Li and Anantharam, 2021). By integrating with deep learning, we provide a new coding scheme for distributed lossy compression with side information at the decoder. We demonstrate the effectiveness of the proposed scheme through experiments involving synthetic Gaussian sources, distributed image compression with MNIST and vertical federated learning with CIFAR-10.} }
Endnote
%0 Conference Paper %T Importance Matching Lemma for Lossy Compression with Side Information %A Buu Phan %A Ashish Khisti %A Christos Louizos %B Proceedings of The 27th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2024 %E Sanjoy Dasgupta %E Stephan Mandt %E Yingzhen Li %F pmlr-v238-phan24a %I PMLR %P 1387--1395 %U https://proceedings.mlr.press/v238/phan24a.html %V 238 %X We propose two extensions to existing importance sampling based methods for lossy compression. First, we introduce an importance sampling based compression scheme that is a variant of ordered random coding (Theis and Ahmed, 2022) and is amenable to direct evaluation of the achievable compression rate for a finite number of samples. Our second and major contribution is the \emph{importance matching lemma}, which is a finite proposal counterpart of the recently introduced {Poisson matching lemma} (Li and Anantharam, 2021). By integrating with deep learning, we provide a new coding scheme for distributed lossy compression with side information at the decoder. We demonstrate the effectiveness of the proposed scheme through experiments involving synthetic Gaussian sources, distributed image compression with MNIST and vertical federated learning with CIFAR-10.
APA
Phan, B., Khisti, A. & Louizos, C.. (2024). Importance Matching Lemma for Lossy Compression with Side Information. Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 238:1387-1395 Available from https://proceedings.mlr.press/v238/phan24a.html.

Related Material