Private and Efficient Federated Statistical Learning

Jaemu Heo, Xiwen Feng, Jeonghun Kang, Taehwan Kim, Changgee Chang
Proceedings of The 29th International Conference on Artificial Intelligence and Statistics, PMLR 300:3853-3861, 2026.

Abstract

Federated Learning (FL) enables collaborative model training across multiple data sources while preserving data privacy, and differential privacy (DP) provides a probabilistic framework to safeguard sensitive information when sharing output derived from data. While numerous DP-FL methods exist, achieving both DP and efficient utility in federated statistical learning remains a significant challenge. In this work, we propose a novel federated statistical learning framework that ensures efficient, robust, and privacy-preserving estimation. We introduce a new noising mechanism that encodes uncertainty along with the maximum likelihood estimate (MLE) by leveraging multiple noisy copies of the MLE. To calibrate noise effectively, we extend the smooth sensitivity to account for data-dependent correlations, ensuring strong DP guarantees while maintaining utility. Additionally, we develop INFEMBLER, an information-assembling algorithm that efficiently de-noises multiple noisy MLE copies using a hierarchical Bayesian model and via an expectation-maximization (EM) algorithm. INFEMBLER significantly enhances estimation efficiency over existing methods and is inherently robust, providing estimates at least as reliable as those derived from local data alone, thereby preserving the benefits of FL. We establish its asymptotic properties and validate its effectiveness through experiments on both simulated and real datasets, demonstrating its superior statistical efficiency and robustness.

Cite this Paper

BibTeX
@InProceedings{pmlr-v300-heo26a, title = { Private and Efficient Federated Statistical Learning }, author = {Heo, Jaemu and Feng, Xiwen and Kang, Jeonghun and Kim, Taehwan and Chang, Changgee}, booktitle = {Proceedings of The 29th International Conference on Artificial Intelligence and Statistics}, pages = {3853--3861}, year = {2026}, editor = {Khan, Emtiyaz and Li, Yingzhen and Solin, Arno and Ramdas, Aaditya}, volume = {300}, series = {Proceedings of Machine Learning Research}, month = {02--05 May}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v300/main/assets/heo26a/heo26a.pdf}, url = {https://proceedings.mlr.press/v300/heo26a.html}, abstract = { Federated Learning (FL) enables collaborative model training across multiple data sources while preserving data privacy, and differential privacy (DP) provides a probabilistic framework to safeguard sensitive information when sharing output derived from data. While numerous DP-FL methods exist, achieving both DP and efficient utility in federated statistical learning remains a significant challenge. In this work, we propose a novel federated statistical learning framework that ensures efficient, robust, and privacy-preserving estimation. We introduce a new noising mechanism that encodes uncertainty along with the maximum likelihood estimate (MLE) by leveraging multiple noisy copies of the MLE. To calibrate noise effectively, we extend the smooth sensitivity to account for data-dependent correlations, ensuring strong DP guarantees while maintaining utility. Additionally, we develop INFEMBLER, an information-assembling algorithm that efficiently de-noises multiple noisy MLE copies using a hierarchical Bayesian model and via an expectation-maximization (EM) algorithm. INFEMBLER significantly enhances estimation efficiency over existing methods and is inherently robust, providing estimates at least as reliable as those derived from local data alone, thereby preserving the benefits of FL. We establish its asymptotic properties and validate its effectiveness through experiments on both simulated and real datasets, demonstrating its superior statistical efficiency and robustness. } }
Endnote
%0 Conference Paper %T Private and Efficient Federated Statistical Learning %A Jaemu Heo %A Xiwen Feng %A Jeonghun Kang %A Taehwan Kim %A Changgee Chang %B Proceedings of The 29th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2026 %E Emtiyaz Khan %E Yingzhen Li %E Arno Solin %E Aaditya Ramdas %F pmlr-v300-heo26a %I PMLR %P 3853--3861 %U https://proceedings.mlr.press/v300/heo26a.html %V 300 %X Federated Learning (FL) enables collaborative model training across multiple data sources while preserving data privacy, and differential privacy (DP) provides a probabilistic framework to safeguard sensitive information when sharing output derived from data. While numerous DP-FL methods exist, achieving both DP and efficient utility in federated statistical learning remains a significant challenge. In this work, we propose a novel federated statistical learning framework that ensures efficient, robust, and privacy-preserving estimation. We introduce a new noising mechanism that encodes uncertainty along with the maximum likelihood estimate (MLE) by leveraging multiple noisy copies of the MLE. To calibrate noise effectively, we extend the smooth sensitivity to account for data-dependent correlations, ensuring strong DP guarantees while maintaining utility. Additionally, we develop INFEMBLER, an information-assembling algorithm that efficiently de-noises multiple noisy MLE copies using a hierarchical Bayesian model and via an expectation-maximization (EM) algorithm. INFEMBLER significantly enhances estimation efficiency over existing methods and is inherently robust, providing estimates at least as reliable as those derived from local data alone, thereby preserving the benefits of FL. We establish its asymptotic properties and validate its effectiveness through experiments on both simulated and real datasets, demonstrating its superior statistical efficiency and robustness.
APA
Heo, J., Feng, X., Kang, J., Kim, T. & Chang, C.. (2026). Private and Efficient Federated Statistical Learning . Proceedings of The 29th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 300:3853-3861 Available from https://proceedings.mlr.press/v300/heo26a.html.

Related Material