Dropout-Resilient Secure Multi-Party Collaborative Learning with Linear Communication Complexity

Xingyu Lu, Hasin Us Sami, Başak Güler
Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, PMLR 206:10566-10593, 2023.

Abstract

Collaborative machine learning enables privacy-preserving training of machine learning models without collecting sensitive client data. Despite recent breakthroughs, communication bottleneck is still a major challenge against its scalability to larger networks. To address this challenge, we propose PICO, the first collaborative learning framework with linear communication complexity, significantly improving over the quadratic state-of-the-art, under formal information-theoretic privacy guarantees. Theoretical analysis demonstrates that PICO slashes the communication cost while achieving equal computational complexity, adversary resilience, robustness to client dropouts, and model accuracy to the state-of-the-art. Extensive experiments demonstrate up to 91x reduction in the communication overhead, and up to 7x speed-up in the wall-clock training time compared to the state-of-the-art. As such, PICO addresses a key technical challenge in multi-party collaborative learning, paving the way for future large-scale privacy-preserving learning frameworks.

Cite this Paper

BibTeX
@InProceedings{pmlr-v206-lu23a, title = {Dropout-Resilient Secure Multi-Party Collaborative Learning with Linear Communication Complexity}, author = {Lu, Xingyu and Sami, Hasin Us and G\"uler, Ba\c{s}ak}, booktitle = {Proceedings of The 26th International Conference on Artificial Intelligence and Statistics}, pages = {10566--10593}, year = {2023}, editor = {Ruiz, Francisco and Dy, Jennifer and van de Meent, Jan-Willem}, volume = {206}, series = {Proceedings of Machine Learning Research}, month = {25--27 Apr}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v206/lu23a/lu23a.pdf}, url = {https://proceedings.mlr.press/v206/lu23a.html}, abstract = {Collaborative machine learning enables privacy-preserving training of machine learning models without collecting sensitive client data. Despite recent breakthroughs, communication bottleneck is still a major challenge against its scalability to larger networks. To address this challenge, we propose PICO, the first collaborative learning framework with linear communication complexity, significantly improving over the quadratic state-of-the-art, under formal information-theoretic privacy guarantees. Theoretical analysis demonstrates that PICO slashes the communication cost while achieving equal computational complexity, adversary resilience, robustness to client dropouts, and model accuracy to the state-of-the-art. Extensive experiments demonstrate up to 91x reduction in the communication overhead, and up to 7x speed-up in the wall-clock training time compared to the state-of-the-art. As such, PICO addresses a key technical challenge in multi-party collaborative learning, paving the way for future large-scale privacy-preserving learning frameworks.} }
Endnote
%0 Conference Paper %T Dropout-Resilient Secure Multi-Party Collaborative Learning with Linear Communication Complexity %A Xingyu Lu %A Hasin Us Sami %A Başak Güler %B Proceedings of The 26th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2023 %E Francisco Ruiz %E Jennifer Dy %E Jan-Willem van de Meent %F pmlr-v206-lu23a %I PMLR %P 10566--10593 %U https://proceedings.mlr.press/v206/lu23a.html %V 206 %X Collaborative machine learning enables privacy-preserving training of machine learning models without collecting sensitive client data. Despite recent breakthroughs, communication bottleneck is still a major challenge against its scalability to larger networks. To address this challenge, we propose PICO, the first collaborative learning framework with linear communication complexity, significantly improving over the quadratic state-of-the-art, under formal information-theoretic privacy guarantees. Theoretical analysis demonstrates that PICO slashes the communication cost while achieving equal computational complexity, adversary resilience, robustness to client dropouts, and model accuracy to the state-of-the-art. Extensive experiments demonstrate up to 91x reduction in the communication overhead, and up to 7x speed-up in the wall-clock training time compared to the state-of-the-art. As such, PICO addresses a key technical challenge in multi-party collaborative learning, paving the way for future large-scale privacy-preserving learning frameworks.
APA
Lu, X., Sami, H.U. & Güler, B.. (2023). Dropout-Resilient Secure Multi-Party Collaborative Learning with Linear Communication Complexity. Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 206:10566-10593 Available from https://proceedings.mlr.press/v206/lu23a.html.

Related Material