DAdaQuant: Doubly-adaptive quantization for communication-efficient Federated Learning

Robert Hönig, Yiren Zhao, Robert Mullins
Proceedings of the 39th International Conference on Machine Learning, PMLR 162:8852-8866, 2022.

Abstract

Federated Learning (FL) is a powerful technique to train a model on a server with data from several clients in a privacy-preserving manner. FL incurs significant communication costs because it repeatedly transmits the model between the server and clients. Recently proposed algorithms quantize the model parameters to efficiently compress FL communication. We find that dynamic adaptations of the quantization level can boost compression without sacrificing model quality. We introduce DAdaQuant as a doubly-adaptive quantization algorithm that dynamically changes the quantization level across time and different clients. Our experiments show that DAdaQuant consistently improves client$\rightarrow$server compression, outperforming the strongest non-adaptive baselines by up to $2.8\times$.

Cite this Paper

BibTeX
@InProceedings{pmlr-v162-honig22a, title = {{DA}da{Q}uant: Doubly-adaptive quantization for communication-efficient Federated Learning}, author = {H{\"o}nig, Robert and Zhao, Yiren and Mullins, Robert}, booktitle = {Proceedings of the 39th International Conference on Machine Learning}, pages = {8852--8866}, year = {2022}, editor = {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan}, volume = {162}, series = {Proceedings of Machine Learning Research}, month = {17--23 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v162/honig22a/honig22a.pdf}, url = {https://proceedings.mlr.press/v162/honig22a.html}, abstract = {Federated Learning (FL) is a powerful technique to train a model on a server with data from several clients in a privacy-preserving manner. FL incurs significant communication costs because it repeatedly transmits the model between the server and clients. Recently proposed algorithms quantize the model parameters to efficiently compress FL communication. We find that dynamic adaptations of the quantization level can boost compression without sacrificing model quality. We introduce DAdaQuant as a doubly-adaptive quantization algorithm that dynamically changes the quantization level across time and different clients. Our experiments show that DAdaQuant consistently improves client$\rightarrow$server compression, outperforming the strongest non-adaptive baselines by up to $2.8\times$.} }
Endnote
%0 Conference Paper %T DAdaQuant: Doubly-adaptive quantization for communication-efficient Federated Learning %A Robert Hönig %A Yiren Zhao %A Robert Mullins %B Proceedings of the 39th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2022 %E Kamalika Chaudhuri %E Stefanie Jegelka %E Le Song %E Csaba Szepesvari %E Gang Niu %E Sivan Sabato %F pmlr-v162-honig22a %I PMLR %P 8852--8866 %U https://proceedings.mlr.press/v162/honig22a.html %V 162 %X Federated Learning (FL) is a powerful technique to train a model on a server with data from several clients in a privacy-preserving manner. FL incurs significant communication costs because it repeatedly transmits the model between the server and clients. Recently proposed algorithms quantize the model parameters to efficiently compress FL communication. We find that dynamic adaptations of the quantization level can boost compression without sacrificing model quality. We introduce DAdaQuant as a doubly-adaptive quantization algorithm that dynamically changes the quantization level across time and different clients. Our experiments show that DAdaQuant consistently improves client$\rightarrow$server compression, outperforming the strongest non-adaptive baselines by up to $2.8\times$.
APA
Hönig, R., Zhao, Y. & Mullins, R.. (2022). DAdaQuant: Doubly-adaptive quantization for communication-efficient Federated Learning. Proceedings of the 39th International Conference on Machine Learning, in Proceedings of Machine Learning Research 162:8852-8866 Available from https://proceedings.mlr.press/v162/honig22a.html.

Related Material