Decentralized Collaborative Learning of Personalized Models over Networks

Paul Vanhaesebrouck, Aurélien Bellet, Marc Tommasi
Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, PMLR 54:509-517, 2017.

Abstract

We consider a set of learning agents in a collaborative peer-to-peer network, where each agent learns a personalized model according to its own learning objective. The question addressed in this paper is: how can agents improve upon their locally trained model by communicating with other agents that have similar objectives? We introduce and analyze two asynchronous gossip algorithms running in a fully decentralized manner. Our first approach, inspired from label propagation, aims to smooth pre-trained local models over the network while accounting for the confidence that each agent has in its initial model. In our second approach, agents jointly learn and propagate their model by making iterative updates based on both their local dataset and the behavior of their neighbors. To optimize this challenging objective, our decentralized algorithm is based on ADMM.

Cite this Paper

BibTeX
@InProceedings{pmlr-v54-vanhaesebrouck17a, title = {{Decentralized Collaborative Learning of Personalized Models over Networks}}, author = {Vanhaesebrouck, Paul and Bellet, Aurélien and Tommasi, Marc}, booktitle = {Proceedings of the 20th International Conference on Artificial Intelligence and Statistics}, pages = {509--517}, year = {2017}, editor = {Singh, Aarti and Zhu, Jerry}, volume = {54}, series = {Proceedings of Machine Learning Research}, month = {20--22 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v54/vanhaesebrouck17a/vanhaesebrouck17a.pdf}, url = {https://proceedings.mlr.press/v54/vanhaesebrouck17a.html}, abstract = {We consider a set of learning agents in a collaborative peer-to-peer network, where each agent learns a personalized model according to its own learning objective. The question addressed in this paper is: how can agents improve upon their locally trained model by communicating with other agents that have similar objectives? We introduce and analyze two asynchronous gossip algorithms running in a fully decentralized manner. Our first approach, inspired from label propagation, aims to smooth pre-trained local models over the network while accounting for the confidence that each agent has in its initial model. In our second approach, agents jointly learn and propagate their model by making iterative updates based on both their local dataset and the behavior of their neighbors. To optimize this challenging objective, our decentralized algorithm is based on ADMM.} }
Endnote
%0 Conference Paper %T Decentralized Collaborative Learning of Personalized Models over Networks %A Paul Vanhaesebrouck %A Aurélien Bellet %A Marc Tommasi %B Proceedings of the 20th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2017 %E Aarti Singh %E Jerry Zhu %F pmlr-v54-vanhaesebrouck17a %I PMLR %P 509--517 %U https://proceedings.mlr.press/v54/vanhaesebrouck17a.html %V 54 %X We consider a set of learning agents in a collaborative peer-to-peer network, where each agent learns a personalized model according to its own learning objective. The question addressed in this paper is: how can agents improve upon their locally trained model by communicating with other agents that have similar objectives? We introduce and analyze two asynchronous gossip algorithms running in a fully decentralized manner. Our first approach, inspired from label propagation, aims to smooth pre-trained local models over the network while accounting for the confidence that each agent has in its initial model. In our second approach, agents jointly learn and propagate their model by making iterative updates based on both their local dataset and the behavior of their neighbors. To optimize this challenging objective, our decentralized algorithm is based on ADMM.
APA
Vanhaesebrouck, P., Bellet, A. & Tommasi, M.. (2017). Decentralized Collaborative Learning of Personalized Models over Networks. Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 54:509-517 Available from https://proceedings.mlr.press/v54/vanhaesebrouck17a.html.

Related Material