Personalized Additive Modeling for Multi-level Federated Learning

Shutong Chen, Guodong Long, Tianyi Zhou, Jie Ma, Jing Jiang, Chengqi Zhang
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:15814-15835, 2026.

Abstract

Contemporary AI faces the challenge of balancing generality with user-specific personalization. In federated learning (FL), this challenge is amplified by highly heterogeneous client data with complex non-IID patterns beyond standard modeling assumptions. Many existing FL methods are designed for relatively restricted heterogeneity settings (e.g., a fixed number of clusters or a fixed form of personalization), limiting their robustness under complex structures. In this work, we study FL from a multi-level non-IID perspective, where client similarity is approximated by multiple granularities of shared knowledge: global, subgroup, and client-specific components. This view captures coarse-to-fine relationships while requiring less prior knowledge of task boundaries. Building on this insight, we propose Federated Multi-level Additive Modeling (FeMAM), which learns multiple levels of shareable models and constructs personalized predictors via additive composition across levels. To move beyond a fixed structure, FeMAM allows models to grow and be pruned dynamically during training, adapting to diverse federated scenarios. Despite employing multiple models, FeMAM remains cost-friendly by activating only a small subset (one level) of models for training at a time. Extensive experiments show that FeMAM effectively approximates complex non-IID structures and consistently outperforms representative clustered and personalized FL baselines.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-chen26cp, title = {Personalized Additive Modeling for Multi-level Federated Learning}, author = {Chen, Shutong and Long, Guodong and Zhou, Tianyi and Ma, Jie and Jiang, Jing and Zhang, Chengqi}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {15814--15835}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/chen26cp/chen26cp.pdf}, url = {https://proceedings.mlr.press/v306/chen26cp.html}, abstract = {Contemporary AI faces the challenge of balancing generality with user-specific personalization. In federated learning (FL), this challenge is amplified by highly heterogeneous client data with complex non-IID patterns beyond standard modeling assumptions. Many existing FL methods are designed for relatively restricted heterogeneity settings (e.g., a fixed number of clusters or a fixed form of personalization), limiting their robustness under complex structures. In this work, we study FL from a multi-level non-IID perspective, where client similarity is approximated by multiple granularities of shared knowledge: global, subgroup, and client-specific components. This view captures coarse-to-fine relationships while requiring less prior knowledge of task boundaries. Building on this insight, we propose Federated Multi-level Additive Modeling (FeMAM), which learns multiple levels of shareable models and constructs personalized predictors via additive composition across levels. To move beyond a fixed structure, FeMAM allows models to grow and be pruned dynamically during training, adapting to diverse federated scenarios. Despite employing multiple models, FeMAM remains cost-friendly by activating only a small subset (one level) of models for training at a time. Extensive experiments show that FeMAM effectively approximates complex non-IID structures and consistently outperforms representative clustered and personalized FL baselines.} }
Endnote
%0 Conference Paper %T Personalized Additive Modeling for Multi-level Federated Learning %A Shutong Chen %A Guodong Long %A Tianyi Zhou %A Jie Ma %A Jing Jiang %A Chengqi Zhang %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-chen26cp %I PMLR %P 15814--15835 %U https://proceedings.mlr.press/v306/chen26cp.html %V 306 %X Contemporary AI faces the challenge of balancing generality with user-specific personalization. In federated learning (FL), this challenge is amplified by highly heterogeneous client data with complex non-IID patterns beyond standard modeling assumptions. Many existing FL methods are designed for relatively restricted heterogeneity settings (e.g., a fixed number of clusters or a fixed form of personalization), limiting their robustness under complex structures. In this work, we study FL from a multi-level non-IID perspective, where client similarity is approximated by multiple granularities of shared knowledge: global, subgroup, and client-specific components. This view captures coarse-to-fine relationships while requiring less prior knowledge of task boundaries. Building on this insight, we propose Federated Multi-level Additive Modeling (FeMAM), which learns multiple levels of shareable models and constructs personalized predictors via additive composition across levels. To move beyond a fixed structure, FeMAM allows models to grow and be pruned dynamically during training, adapting to diverse federated scenarios. Despite employing multiple models, FeMAM remains cost-friendly by activating only a small subset (one level) of models for training at a time. Extensive experiments show that FeMAM effectively approximates complex non-IID structures and consistently outperforms representative clustered and personalized FL baselines.
APA
Chen, S., Long, G., Zhou, T., Ma, J., Jiang, J. & Zhang, C.. (2026). Personalized Additive Modeling for Multi-level Federated Learning. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:15814-15835 Available from https://proceedings.mlr.press/v306/chen26cp.html.

Related Material