Model Fusion for Personalized Learning

Thanh Chi Lam, Nghia Hoang, Bryan Kian Hsiang Low, Patrick Jaillet
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:5948-5958, 2021.

Abstract

Production systems operating on a growing domain of analytic services often require generating warm-start solution models for emerging tasks with limited data. One potential approach to address this warm-start challenge is to adopt meta learning to generate a base model that can be adapted to solve unseen tasks with minimal fine-tuning. This however requires the training processes of previous solution models of existing tasks to be synchronized. This is not possible if these models were pre-trained separately on private data owned by different entities and cannot be synchronously re-trained. To accommodate for such scenarios, we develop a new personalized learning framework that synthesizes customized models for unseen tasks via fusion of independently pre-trained models of related tasks. We establish performance guarantee for the proposed framework and demonstrate its effectiveness on both synthetic and real datasets.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-lam21a, title = {Model Fusion for Personalized Learning}, author = {Lam, Thanh Chi and Hoang, Nghia and Low, Bryan Kian Hsiang and Jaillet, Patrick}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {5948--5958}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/lam21a/lam21a.pdf}, url = {https://proceedings.mlr.press/v139/lam21a.html}, abstract = {Production systems operating on a growing domain of analytic services often require generating warm-start solution models for emerging tasks with limited data. One potential approach to address this warm-start challenge is to adopt meta learning to generate a base model that can be adapted to solve unseen tasks with minimal fine-tuning. This however requires the training processes of previous solution models of existing tasks to be synchronized. This is not possible if these models were pre-trained separately on private data owned by different entities and cannot be synchronously re-trained. To accommodate for such scenarios, we develop a new personalized learning framework that synthesizes customized models for unseen tasks via fusion of independently pre-trained models of related tasks. We establish performance guarantee for the proposed framework and demonstrate its effectiveness on both synthetic and real datasets.} }
Endnote
%0 Conference Paper %T Model Fusion for Personalized Learning %A Thanh Chi Lam %A Nghia Hoang %A Bryan Kian Hsiang Low %A Patrick Jaillet %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-lam21a %I PMLR %P 5948--5958 %U https://proceedings.mlr.press/v139/lam21a.html %V 139 %X Production systems operating on a growing domain of analytic services often require generating warm-start solution models for emerging tasks with limited data. One potential approach to address this warm-start challenge is to adopt meta learning to generate a base model that can be adapted to solve unseen tasks with minimal fine-tuning. This however requires the training processes of previous solution models of existing tasks to be synchronized. This is not possible if these models were pre-trained separately on private data owned by different entities and cannot be synchronously re-trained. To accommodate for such scenarios, we develop a new personalized learning framework that synthesizes customized models for unseen tasks via fusion of independently pre-trained models of related tasks. We establish performance guarantee for the proposed framework and demonstrate its effectiveness on both synthetic and real datasets.
APA
Lam, T.C., Hoang, N., Low, B.K.H. & Jaillet, P.. (2021). Model Fusion for Personalized Learning. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:5948-5958 Available from https://proceedings.mlr.press/v139/lam21a.html.

Related Material