Algorithms for the Communication of Samples

Lucas Theis, Noureldin Y Ahmed
Proceedings of the 39th International Conference on Machine Learning, PMLR 162:21308-21328, 2022.

Abstract

The efficient communication of noisy data has applications in several areas of machine learning, such as neural compression or differential privacy, and is also known as reverse channel coding or the channel simulation problem. Here we propose two new coding schemes with practical advantages over existing approaches. First, we introduce ordered random coding (ORC) which uses a simple trick to reduce the coding cost of previous approaches. This scheme further illuminates a connection between schemes based on importance sampling and the so-called Poisson functional representation. Second, we describe a hybrid coding scheme which uses dithered quantization to more efficiently communicate samples from distributions with bounded support.

Cite this Paper

BibTeX
@InProceedings{pmlr-v162-theis22a, title = {Algorithms for the Communication of Samples}, author = {Theis, Lucas and Ahmed, Noureldin Y}, booktitle = {Proceedings of the 39th International Conference on Machine Learning}, pages = {21308--21328}, year = {2022}, editor = {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan}, volume = {162}, series = {Proceedings of Machine Learning Research}, month = {17--23 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v162/theis22a/theis22a.pdf}, url = {https://proceedings.mlr.press/v162/theis22a.html}, abstract = {The efficient communication of noisy data has applications in several areas of machine learning, such as neural compression or differential privacy, and is also known as reverse channel coding or the channel simulation problem. Here we propose two new coding schemes with practical advantages over existing approaches. First, we introduce ordered random coding (ORC) which uses a simple trick to reduce the coding cost of previous approaches. This scheme further illuminates a connection between schemes based on importance sampling and the so-called Poisson functional representation. Second, we describe a hybrid coding scheme which uses dithered quantization to more efficiently communicate samples from distributions with bounded support.} }
Endnote
%0 Conference Paper %T Algorithms for the Communication of Samples %A Lucas Theis %A Noureldin Y Ahmed %B Proceedings of the 39th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2022 %E Kamalika Chaudhuri %E Stefanie Jegelka %E Le Song %E Csaba Szepesvari %E Gang Niu %E Sivan Sabato %F pmlr-v162-theis22a %I PMLR %P 21308--21328 %U https://proceedings.mlr.press/v162/theis22a.html %V 162 %X The efficient communication of noisy data has applications in several areas of machine learning, such as neural compression or differential privacy, and is also known as reverse channel coding or the channel simulation problem. Here we propose two new coding schemes with practical advantages over existing approaches. First, we introduce ordered random coding (ORC) which uses a simple trick to reduce the coding cost of previous approaches. This scheme further illuminates a connection between schemes based on importance sampling and the so-called Poisson functional representation. Second, we describe a hybrid coding scheme which uses dithered quantization to more efficiently communicate samples from distributions with bounded support.
APA
Theis, L. & Ahmed, N.Y.. (2022). Algorithms for the Communication of Samples. Proceedings of the 39th International Conference on Machine Learning, in Proceedings of Machine Learning Research 162:21308-21328 Available from https://proceedings.mlr.press/v162/theis22a.html.

Related Material